Nov 02 02:44:55.197798 ip-10-0-131-112 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 02 02:44:55.197809 ip-10-0-131-112 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 02 02:44:55.197816 ip-10-0-131-112 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 02 02:44:55.198036 ip-10-0-131-112 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 02 02:45:06.660858 ip-10-0-131-112 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 02 02:45:06.660873 ip-10-0-131-112 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5c03ba59a2534d0baaa1190bf2694a69 --
Nov 02 02:46:55.933915 ip-10-0-131-112 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 02:46:56.365481 ip-10-0-131-112 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 02:46:56.365481 ip-10-0-131-112 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 02:46:56.365481 ip-10-0-131-112 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 02:46:56.365481 ip-10-0-131-112 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 02:46:56.370458 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370369    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:46:56.370458 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370454    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:46:56.370458 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370459    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:46:56.370458 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370462    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:46:56.370458 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370465    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:46:56.370458 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370468    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370471    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370474    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370476    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370479    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370481    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370484    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370488    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370490    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370492    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370495    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370498    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370500    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370503    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370506    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370508    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370511    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370513    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370515    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370518    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:46:56.370698 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370520    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370523    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370525    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370528    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370531    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370533    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370539    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370542    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370545    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370547    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370549    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370552    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370555    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370557    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370561    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370564    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370566    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370569    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370571    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370574    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:46:56.371169 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370576    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370579    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370581    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370584    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370586    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370590    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370593    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370595    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370598    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370600    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370603    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370606    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370610    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370615    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370618    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370637    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370639    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370642    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370645    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:46:56.371653 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370648    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370652    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370656    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370659    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370662    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370665    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370667    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370670    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370673    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370676    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370679    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370682    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370684    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370687    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370690    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370692    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370695    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370698    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370701    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:46:56.372112 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370704    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370707    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370710    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370712    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370715    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370718    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370721    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370724    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370726    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370729    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370731    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370733    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370736    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370738    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370741    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370743    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370746    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370748    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370751    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370753    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:46:56.372561 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370757    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370759    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.370762    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371526    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371534    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371537    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371540    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371543    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371545    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371548    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371550    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371553    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371556    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371559    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371562    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371565    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371568    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371571    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371573    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371576    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:46:56.373075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371579    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371581    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371584    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371586    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371589    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371591    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371594    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371596    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371599    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371601    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371604    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371606    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371609    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371612    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371614    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371617    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371643    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371647    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371649    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:46:56.373564 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371652    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371655    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371659    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371663    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371667    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371670    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371673    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371676    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371680    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371683    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371686    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371689    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371691    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371694    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371696    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371699    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371701    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371704    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371706    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:46:56.374037 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371709    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371711    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371714    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371716    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371719    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371721    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371724    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371727    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371729    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371731    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371734    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371736    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371739    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371741    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371744    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371747    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371749    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371752    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371755    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:46:56.374490 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371757    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371759    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371763    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371766    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371769    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371771    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371774    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371776    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371778    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371781    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371783    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371786    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371789    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371791    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371794    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371796    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371799    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371801    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371803    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371806    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:46:56.374949 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371808    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371811    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371813    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371816    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371818    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371821    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371824    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371827    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371829    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371832    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371834    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.371837    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372336    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372341    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372344    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372348    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372350    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372353    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372356    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372359    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:46:56.375442 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372361    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372364    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372366    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372369    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372373    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372375    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372378    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372380    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372383    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372385    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372388    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372390    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372393    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372395    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372398    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372400    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372403    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372405    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372407    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372410    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:46:56.375942 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372413    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372416    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372418    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372421    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372423    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372426    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372428    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372431    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372434    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372436    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372439    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372441    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372444    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372446    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372449    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372451    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372454    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372456    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372459    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372461    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:46:56.376422 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372464    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372466    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372469    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372471    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372473    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372476    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372478    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372481    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372483    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372485    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372488    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372491    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372493    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372496    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372498    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372501    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372504    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372506    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372509    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:46:56.376906 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372514    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372517    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372520    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372522    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372525    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372527    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372530    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372532    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372535    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372537    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372539    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372542    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372544    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372547    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372549    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372551    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372554    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372556    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372558    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372562    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:46:56.377380 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372565    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372568    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372570    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372575    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372580    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372584    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372587    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372590    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372592    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372595    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372597    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372600    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372604    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372606    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372610    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372612    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372615    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372634    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.372638    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:46:56.377854 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373599    2574 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373609    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373617    2574 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373635    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373640    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373643    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373647    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373652    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373655    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373658    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373661    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373665    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373668    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373670    2574 flags.go:64] FLAG: --cgroup-root=""
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373674    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373677    2574 flags.go:64] FLAG: --client-ca-file=""
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373679    2574 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373683    2574 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373687    2574 flags.go:64] FLAG: --cluster-domain=""
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373689    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373693    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373696    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373699    2574 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373703    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 02:46:56.378306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373705    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373709    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373712    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373715    2574 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373719    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373722    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373725    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373728    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373732    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373735    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373737    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373740    2574 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373744    2574 flags.go:64] FLAG: --enable-server="true"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373746    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373750    2574 flags.go:64] FLAG: --event-burst="100"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373754    2574 flags.go:64] FLAG: --event-qps="50"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373757    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373759    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373762    2574 flags.go:64] FLAG: --eviction-hard=""
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373766    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373769    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373772    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373775    2574 flags.go:64] FLAG: --eviction-soft=""
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373778    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373781    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 02:46:56.378869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373784    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373787    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373790    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373792    2574 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373795    2574 flags.go:64] FLAG: --feature-gates=""
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373799    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373802    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373805    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373808    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373811    2574 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373814    2574 flags.go:64] FLAG: --help="false"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373817    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373820    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373823    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373826    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373829    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373832    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373835    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373838    2574 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373840    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373843    2574 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373846    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373849    2574 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373851    2574 flags.go:64] FLAG: --kube-reserved=""
Nov 02 02:46:56.379478 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373854    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373857    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373860    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373862    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373865    2574 flags.go:64] FLAG: --lock-file=""
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373868    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373871    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373874    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373879    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373882    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373885    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373888    2574 flags.go:64] FLAG: --logging-format="text"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373891    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373895    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373897    2574 flags.go:64] FLAG: --manifest-url=""
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373900    2574 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373904    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373908    2574 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373912    2574 flags.go:64] FLAG: --max-pods="110"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373915    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373917    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373921    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373924    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373927    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373930    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 02:46:56.380150 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373933    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373940    2574 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373943    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373946    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373949    2574 flags.go:64] FLAG: --pod-cidr=""
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373952    2574 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373954    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373958    2574 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373961    2574 flags.go:64] FLAG: --port="10250"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373964    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373966    2574 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-016326884fc7c9386"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373969    2574 flags.go:64] FLAG: --qos-reserved=""
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373972    2574 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373975    2574 flags.go:64] FLAG: --register-node="true"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373978    2574 flags.go:64] FLAG: --register-with-taints=""
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373982    2574 flags.go:64] FLAG: --registry-burst="10"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373984    2574 flags.go:64] FLAG: --registry-qps="5"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373987    2574 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373990    2574 flags.go:64] FLAG: --reserved-memory=""
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373993    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373996    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.373999    2574 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374002    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374005    2574 flags.go:64] FLAG: --runonce="false"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374008    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374011    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374014    2574 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 02:46:56.380812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374017    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374019    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374023    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374026    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374031    2574 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374034    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374036    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374040    2574 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374043    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374046    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374049    2574 flags.go:64] FLAG: --system-cgroups=""
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374052    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374059    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374061    2574 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374064    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374068    2574 flags.go:64] FLAG: --tls-min-version=""
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374071    2574 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374074    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374077    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374080    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374083    2574 flags.go:64] FLAG: --v="2"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374087    2574 flags.go:64] FLAG: --version="false"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374091    2574 flags.go:64] FLAG: --vmodule=""
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374095    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374098    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 02:46:56.381440 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374234    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374238    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374242    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374244    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374247    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374250    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374253    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374255    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374258    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374261    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374268    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374273    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374277    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374281    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374286    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374290    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374295    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374300    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374303    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:46:56.382065 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374306    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374309    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374311    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374314    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374317    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374319    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374322    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374325    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374327    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374330    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374332    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374335    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374337    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374340    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374342    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374344    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374347    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374349    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374352    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374354    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:46:56.382927 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374357    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374359    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374361    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374364    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374369    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374371    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374374    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374377    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374381    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374384    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374387    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374390    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374394    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374397    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374399    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374402    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374404    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374407    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374409    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374412    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:46:56.383396 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374414    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374417    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374419    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374422    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374424    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374427    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374429    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374431    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374434    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374436    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374439    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374441    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374444    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374446    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374448    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374451    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374454    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374457    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374460    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374462    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:46:56.383886 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374465    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374467    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374470    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374472    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374476    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374479    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374481    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374483    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374486    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374488    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374491    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374494    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374496    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374499    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374502    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374504    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374507    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374509    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374512    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:46:56.384368 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374514    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374517    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374521    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374524    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374527    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374530    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374532    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374535    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.374562    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374614    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374632    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374639    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374643    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374645    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374648    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:46:56.384841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374650    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374653    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374657    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374660    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374664    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374667    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374670    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374673    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374676    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374679    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374681    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374684    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374687    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374689    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374692    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374695    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374697    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374700    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374702    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374705    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:46:56.385191 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374707    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374709    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374712    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374714    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374717    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374719    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374722    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374725    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374728    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374731    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374733    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374736    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374738    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374741    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374744    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374747    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374749    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374752    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374754    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374761    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:46:56.385743 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374764    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374766    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374769    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374771    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374774    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374776    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374778    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374781    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374783    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374786    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374788    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374791    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374794    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374796    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374799    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374801    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374804    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374807    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374811    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:46:56.386215 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374815    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374818    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374821    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374824    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374827    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374829    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374832    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374835    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374839    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374841    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374844    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374846    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374849    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374851    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374854    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374857    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374859    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374862    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374865    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:46:56.386731 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374867    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374870    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374872    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374875    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374877    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374880    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374882    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374884    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374887    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374889    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374892    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374908    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374912    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374917    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374919    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374922    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374924    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374927    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374930    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374932    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:46:56.387168 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374936    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.374938    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.381447    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.381472    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.381491    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.381496    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381551    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381557    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381560    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381563    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381565    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381568    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381571    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381573    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381577    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381579    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381582    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381585    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381587    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:46:56.387655 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381590    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381593    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381595    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381598    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381600    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381603    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381605    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381608    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381612    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381615    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381618    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381637    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381640    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381642    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381645    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381648    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381652    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381655    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381657    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:46:56.388098 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381660    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381663    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381665    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381669    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381672    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381675    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381678    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381680    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381683    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381686    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381688    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381691    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381693    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381696    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381698    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381701    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381704    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381707    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381709    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:46:56.388544 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381712    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381714    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381717    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381719    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381721    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381724    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381726    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381729    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381732    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381734    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381737    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381740    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381743    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381745    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381748    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381750    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381753    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381755    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381758    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381760    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:46:56.389040 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381763    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381766    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381768    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381770    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381773    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381775    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381777    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381780    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381782    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381785    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381788    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381790    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381793    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381795    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381798    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381800    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381802    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381805    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381807    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381810    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381813    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:46:56.389513 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381815    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381819    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381821    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381824    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381827    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381829    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381832    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381834    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381837    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381839    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381841    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381844    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381846    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.381849    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.381876    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382361    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:46:56.390183 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382366    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382369    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382371    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382375    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382380    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382383    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382386    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382389    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382391    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382394    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382398    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382401    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382404    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382407    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382410    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382413    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382416    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382418    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382421    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:46:56.390808 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382423    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382426    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382429    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382431    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382434    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382436    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382439    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382441    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382443    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382446    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382448    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382451    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382453    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382456    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382458    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382461    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382463    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382466    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382468    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382471    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:46:56.391373 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382473    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382476    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382478    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382480    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382483    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382485    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382488    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382490    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382493    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382495    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382498    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382500    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382503    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382506    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382508    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382511    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382513    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382516    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382518    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:46:56.392012 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382520    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382523    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382525    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382528    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382530    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382533    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382535    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382538    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382540    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382543    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382545    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382548    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382550    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382553    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382555    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382558    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382560    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382563    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382565    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382568    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:46:56.392601 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382570    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382573    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382576    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382579    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382582    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382584    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382587    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382589    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382592    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382594    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382597    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382599    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382602    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382604    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382607    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382609    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382612    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382614    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382616    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382630    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:46:56.393171 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382633    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382635    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382638    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382640    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382643    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382645    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382647    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.382672    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382702    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382706    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382710    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382713    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382716    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382719    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382722    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382724    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:46:56.393837 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382727    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382731    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382734    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382737    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382739    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382742    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382745    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382747    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382750    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382752    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382754    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382757    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382759    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382761    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382764    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382766    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382769    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382771    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382774    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:46:56.394399 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382776    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382778    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382781    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382783    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382786    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382788    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382791    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382793    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382795    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382798    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382800    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382803    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382805    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382808    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382810    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382812    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382815    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382817    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382820    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382822    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:46:56.394875 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382825    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382827    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382830    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382832    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382835    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382837    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382839    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382842    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382844    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382847    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382849    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382852    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382854    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382856    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382859    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382861    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382864    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382866    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382868    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:46:56.395354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382871    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382873    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382876    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382878    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382880    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382883    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382885    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382888    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382890    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382892    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382895    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382898    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382900    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382906    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382909    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382911    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382914    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382917    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382919    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382922    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:46:56.396075 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382924    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382927    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382930    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382932    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382935    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382938    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382940    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382943    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382945    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382948    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382950    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382952    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382955    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382957    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382959    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382962    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382964    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382967    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382970    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:56.382972    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:46:56.396574 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.383026    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 02:46:56.397055 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.386254    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 02 02:46:56.397055 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.387094    2574 server.go:1014] "Starting client certificate rotation"
Nov 02 02:46:56.397055 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.387193    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 02:46:56.397055 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.387225    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 02:46:56.397055 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.388879    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 02:46:56.397055 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.392472    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 02:46:56.405180 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.405164    2574 log.go:25] "Validated CRI v1 runtime API"
Nov 02 02:46:56.408155 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.408135    2574 log.go:25] "Validated CRI v1 image API"
Nov 02 02:46:56.409657 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.409611    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 02:46:56.412956 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.412938    2574 fs.go:135] Filesystem UUIDs: map[0c96eaaa-0794-4dc5-8ffc-e9cd62359b22:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c50fc222-72a5-44f0-9c74-d7383b9840aa:/dev/nvme0n1p3]
Nov 02 02:46:56.413004 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.412957    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 02 02:46:56.418021 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.417912    2574 manager.go:217] Machine: {Timestamp:2026-11-02 02:46:56.416601747 +0000 UTC m=+0.360098049 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100512 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208a7391c333b55d569b729662989f SystemUUID:ec208a73-91c3-33b5-5d56-9b729662989f BootID:5c03ba59-a253-4d0b-aaa1-190bf2694a69 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3d:e5:80:1a:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3d:e5:80:1a:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:32:e9:59:3b:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 02:46:56.418143 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.418024    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 02:46:56.418143 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.418108    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 02:46:56.421021 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.420994    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 02:46:56.421163 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.421024    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-112.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 02:46:56.421676 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.421665    2574 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 02:46:56.421713 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.421679    2574 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 02:46:56.421713 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.421709    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 02:46:56.421766 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.421722    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 02:46:56.422654 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.422645    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 02:46:56.426679 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.426668    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 02:46:56.426802 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.426792    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 02:46:56.427557 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.427548    2574 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 02:46:56.427587 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.427562    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 02:46:56.427587 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.427572    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 02:46:56.427587 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.427581    2574 kubelet.go:405] "Adding apiserver pod source"
Nov 02 02:46:56.427670 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.427589    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 02:46:56.427972 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.427960    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 02:46:56.430636 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.430602    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 02:46:56.437360 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.437344    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 02:46:56.438225 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.438212    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 02:46:56.438276 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.438245    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 02:46:56.438396 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.438387    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 02:46:56.438425 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.438403    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 02:46:56.438425 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.438409    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 02:46:56.438425 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.438414    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 02:46:56.438425 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.438420    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 02:46:56.438425 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.438426    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 02:46:56.439153 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.439144    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 02:46:56.439183 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.439154    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 02:46:56.439183 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.439162    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 02:46:56.439183 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.439167    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 02:46:56.439183 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.439176    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 02:46:56.439183 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.439184    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 02:46:56.440611 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.440602    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 02:46:56.440663 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.440612    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 02:46:56.444463 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.444450    2574 server.go:1263] "Started kubelet"
Nov 02 02:46:56.444636 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.444555    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 02:46:56.444636 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.444545    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 02:46:56.444763 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.444672    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 02:46:56.445288 ip-10-0-131-112 systemd[1]: Started Kubernetes Kubelet.
Nov 02 02:46:56.445982 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.445941    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 02:46:56.447070 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.447052    2574 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 02:46:56.451009 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.450987    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 02 02:46:56.452155 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.452137    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 02:46:56.453103 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.453084    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 02:46:56.453191 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.453172    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-112.us-west-1.compute.internal\" not found"
Nov 02 02:46:56.453748 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.453729    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 02:46:56.453748 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.453749    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 02:46:56.453884 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.453727    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 02:46:56.453884 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.453879    2574 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 02:46:56.453946 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.453893    2574 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 02:46:56.453990 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.453971    2574 factory.go:55] Registering systemd factory
Nov 02 02:46:56.453990 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.453982    2574 factory.go:223] Registration of the systemd container factory successfully
Nov 02 02:46:56.454183 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.454170    2574 factory.go:153] Registering CRI-O factory
Nov 02 02:46:56.454240 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.454184    2574 factory.go:223] Registration of the crio container factory successfully
Nov 02 02:46:56.454286 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.454275    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 02:46:56.454327 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.454291    2574 factory.go:103] Registering Raw factory
Nov 02 02:46:56.454327 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.454301    2574 manager.go:1199] Started watching for new ooms in manager
Nov 02 02:46:56.455262 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.455247    2574 manager.go:319] Starting recovery of all containers
Nov 02 02:46:56.464850 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.464829    2574 manager.go:324] Recovery completed
Nov 02 02:46:56.470276 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.470214    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:46:56.471807 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.471784    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:46:56.471870 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.471823    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:46:56.471870 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.471837    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:46:56.472376 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.472361    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:46:56.472376 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.472365    2574 cpu_manager.go:226] "Starting" policy="none"
Nov 02 02:46:56.472376 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.472377    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 02:46:56.472502 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.472394    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 02:46:56.473003 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.472991    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:46:56.473056 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.473014    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:46:56.473056 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.473024    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:46:56.474615 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.474602    2574 policy_none.go:50] "Start"
Nov 02 02:46:56.474686 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.474676    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 02:46:56.474719 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.474691    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 02:46:56.476648 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.476638    2574 policy_none.go:44] "Start"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.521672    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.524375    2574 manager.go:341] "Starting Device Plugin manager"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.524402    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.524411    2574 server.go:87] "Starting device plugin registration server"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.524684    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.524698    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.524834    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.524952    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.524959    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.525494    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.525556    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-112.us-west-1.compute.internal\" not found"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.533890    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.535151    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.535167    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.535182    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 02:46:56.539567 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.535209    2574 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 02 02:46:56.582576 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.582537    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-112.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 02 02:46:56.582576 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.582556    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-112.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 02 02:46:56.583532 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.582553    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-112.us-west-1.compute.internal.18e41a6f2fb5936f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-112.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-112.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 02:46:56.444420975 +0000 UTC m=+0.387917276,LastTimestamp:2026-11-02 02:46:56.444420975 +0000 UTC m=+0.387917276,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-112.us-west-1.compute.internal,}"
Nov 02 02:46:56.625804 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.625741    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:46:56.626645 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.626609    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:46:56.626737 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.626654    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:46:56.626737 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.626664    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:46:56.626737 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.626689    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.636030 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.636003    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"]
Nov 02 02:46:56.636114 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.636064    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:46:56.636839 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.636822    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:46:56.636925 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.636856    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:46:56.636925 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.636868    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:46:56.639151 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.639137    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:46:56.639360 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.639342    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.639421 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.639379    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:46:56.639764 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.639750    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:46:56.639834 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.639771    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:46:56.639834 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.639786    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:46:56.640154 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.640140    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:46:56.640245 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.640170    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:46:56.640245 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.640196    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:46:56.642590 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.642574    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.642668 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.642607    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:46:56.643374 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.643359    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:46:56.643448 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.643383    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:46:56.643448 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.643395    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:46:56.655791 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.655775    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.655846 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.655798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.655846 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.655829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fdb79d33d08466df57752003cd13435-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"9fdb79d33d08466df57752003cd13435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.665016 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.664995    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 02:46:56.665085 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.665042    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2t96f"
Nov 02 02:46:56.674472 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.674450    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-112.us-west-1.compute.internal\" not found" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.677047 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.677030    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-112.us-west-1.compute.internal\" not found" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.756946 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.756913    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.757111 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.756944    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fdb79d33d08466df57752003cd13435-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"9fdb79d33d08466df57752003cd13435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.757111 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.756989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.757111 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.757009    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.757111 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.757026    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fdb79d33d08466df57752003cd13435-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"9fdb79d33d08466df57752003cd13435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.757111 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.757034    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.836709 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.836684    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:46:56.836856 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.836672    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:46:56.838473 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.838456    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:46:56.838963 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.838946    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:46:56.839415 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.839396    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.839489 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:56.839423    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-112.us-west-1.compute.internal\": node \"ip-10-0-131-112.us-west-1.compute.internal\" not found"
Nov 02 02:46:56.854010 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.853987    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.913566 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.913484    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 02:46:56.914448 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.914434    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.969926 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.969902    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 02:46:56.979027 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.979000    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:56.981175 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:56.981158    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:46:57.387638 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.387437    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 02 02:46:57.388147 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.387798    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 02:46:57.388147 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.387845    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 02:46:57.388147 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.387857    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 02:46:57.388147 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.387861    2574 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://af3a1c6fd9c4d43e7823e9d8be5f460a-a3480c90fef6e68b.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-131-112.us-west-1.compute.internal.18e41a6f3157fb53\": read tcp 10.0.131.112:33676->3.15.64.185:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-131-112.us-west-1.compute.internal.18e41a6f3157fb53  default   4946 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-112.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-112.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-112.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 02:46:56 +0000 UTC,LastTimestamp:2026-11-02 02:46:56.62666826 +0000 UTC m=+0.570164560,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-112.us-west-1.compute.internal,}"
Nov 02 02:46:57.428634 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.428600    2574 apiserver.go:52] "Watching apiserver"
Nov 02 02:46:57.453208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.453191    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 02 02:46:57.670987 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.670883    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 02:41:56 +0000 UTC" deadline="2028-04-16 11:59:31.012444708 +0000 UTC"
Nov 02 02:46:57.670987 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.670929    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12753h12m33.341518772s"
Nov 02 02:46:57.688869 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.688847    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 02:46:57.689180 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.689160    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-z7p4b","openshift-image-registry/node-ca-mdddz","openshift-network-diagnostics/network-check-target-2h58d","openshift-network-operator/iptables-alerter-fbwhk","openshift-multus/multus-additional-cni-plugins-2rm9b","openshift-multus/multus-bvqdp","openshift-ovn-kubernetes/ovnkube-node-4bpfv","openshift-cluster-node-tuning-operator/tuned-q8k8l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal","openshift-multus/network-metrics-daemon-4xrlc","kube-system/global-pull-secret-syncer-zs4kx","kube-system/konnectivity-agent-b5wvk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd","kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"]
Nov 02 02:46:57.691665 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.691646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.693704 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.693687    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.695697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.695679    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:46:57.695773 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.695733    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:46:57.697785 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.697769    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.699926 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.699905    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.701994 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.701978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.704210 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.704197    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.706347 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.706327    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.708484 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.708467    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:57.708569 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.708532    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:46:57.710670 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.710654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.710716 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.710705    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:46:57.712861 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.712847    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:46:57.714960 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.714944    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.742895 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.742868    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 02:46:57.742895 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.742894    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 02:46:57.743032 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.742868    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5mjxc\""
Nov 02 02:46:57.744254 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.744241    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 02:46:57.744465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.744450    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 02:46:57.744532 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.744472    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 02:46:57.744727 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.744714    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5v9mr\""
Nov 02 02:46:57.748652 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.748618    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 02:46:57.748708 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.748655    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ghkrt\""
Nov 02 02:46:57.748708 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.748698    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 02:46:57.748820 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.748772    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 02:46:57.750866 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.750852    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cjmh9\""
Nov 02 02:46:57.751184 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.751169    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 02:46:57.751313 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.751298    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 02:46:57.751367 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.751322    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 02:46:57.751367 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.751336    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 02:46:57.751367 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.751338    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 02:46:57.752551 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.752537    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 02:46:57.752749 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.752736    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-pbbl9\""
Nov 02 02:46:57.755377 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.755363    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 02:46:57.762783 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.762764    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-conf\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.762872 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.762791    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg7cc\" (UniqueName: \"kubernetes.io/projected/325964e5-52a3-4672-bda2-ed01ffeee604-kube-api-access-hg7cc\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.762872 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.762806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb902c0e-8074-4363-90c5-04eaa2c82fb3-host\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.762872 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.762821    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2037171a-040e-4b0e-a635-a0b500821338-iptables-alerter-script\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.762872 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.762841    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-netd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.762993 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.762920    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-config\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.762993 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.762937    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-dbus\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.762993 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.762969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjzw\" (UniqueName: \"kubernetes.io/projected/7f3cbd14-2c73-4682-97d1-9beb1df2007e-kube-api-access-lsjzw\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.763076 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-slash\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.763109 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-bin\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.763109 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763099    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-env-overrides\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.763204 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-host\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.763204 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763172    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-tuned\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.763387 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763207    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-system-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.763387 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763234    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-cni-binary-copy\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.763387 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763279    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-etc-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.763387 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.763387 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.763387 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763341    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysconfig\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.763387 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-sys\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.763697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763398    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-kubelet-config\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.763697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:46:57.763697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-kubelet\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.763697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763525    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25h4m\" (UniqueName: \"kubernetes.io/projected/7ffe7d41-e27f-4373-8117-c49008015a88-kube-api-access-25h4m\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.763697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763542    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-var-lib-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.763697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763577    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9913c84-a163-4a5b-b169-cc80806bc03a-ovn-node-metrics-cert\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.763697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763639    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-run\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.763697 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763662    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-tmp\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763732    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-sys-fs\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763772    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-cnibin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-etc-kubernetes\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/325964e5-52a3-4672-bda2-ed01ffeee604-hosts-file\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763856    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/325964e5-52a3-4672-bda2-ed01ffeee604-tmp-dir\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9znzz\" (UniqueName: \"kubernetes.io/projected/1c995359-9608-4568-947c-b9464cf3c4c0-kube-api-access-9znzz\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763930    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee88ccb5-4a04-431a-8963-7b8c7393604a-konnectivity-ca\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:46:57.763968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763952    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-multus\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763970    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-multus-daemon-config\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.763996    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-multus-certs\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764019    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-var-lib-kubelet\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764034    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwzpc\" (UniqueName: \"kubernetes.io/projected/8b6ec759-f178-45cc-9c8e-7f5982c233dc-kube-api-access-vwzpc\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764072    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97v74\" (UniqueName: \"kubernetes.io/projected/fb902c0e-8074-4363-90c5-04eaa2c82fb3-kube-api-access-97v74\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.764208 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-system-cni-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764236    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-hostroot\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764253    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-kubelet\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764368    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764384    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-registration-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764398    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cnibin\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764412    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-bin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764433    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-systemd-units\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.764465 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764462    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-netns\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764505    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-log-socket\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764533    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764548    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-lib-modules\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764561    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-node-log\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-systemd\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764606    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764640    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee88ccb5-4a04-431a-8963-7b8c7393604a-agent-certs\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764671    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-netns\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764690    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-conf-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764704    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-ovn\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764730    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-socket-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764752    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhjgx\" (UniqueName: \"kubernetes.io/projected/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kube-api-access-mhjgx\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764769    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtmk2\" (UniqueName: \"kubernetes.io/projected/2037171a-040e-4b0e-a635-a0b500821338-kube-api-access-dtmk2\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.764793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-socket-dir-parent\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764809    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-script-lib\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764824    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-kubernetes\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-systemd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764859    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764886    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-device-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764914    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fb902c0e-8074-4363-90c5-04eaa2c82fb3-serviceca\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2037171a-040e-4b0e-a635-a0b500821338-host-slash\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764956    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-k8s-cni-cncf-io\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764973    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nhqj\" (UniqueName: \"kubernetes.io/projected/f9913c84-a163-4a5b-b169-cc80806bc03a-kube-api-access-9nhqj\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.765172 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.764986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-modprobe-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.789557 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.789490    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 02:46:57.789557 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.789519    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7jkxs\""
Nov 02 02:46:57.789557 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.789543    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 02:46:57.789808 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.789793    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 02:46:57.789863 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.789812    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 02:46:57.789863 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.789839    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-b8q6l\""
Nov 02 02:46:57.791193 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.791180    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 02:46:57.791949 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.791932    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 02:46:57.792035 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.791962    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 02:46:57.792035 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.791968    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 02:46:57.797992 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.797978    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 02:46:57.798061 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.798050    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-nvmsr\""
Nov 02 02:46:57.798137 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.798124    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 02:46:57.800380 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.800362    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 02:46:57.800430 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.800411    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cgnbb\""
Nov 02 02:46:57.800604 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.800588    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 02:46:57.800691 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.800677    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nfgmr"
Nov 02 02:46:57.842864 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.842845    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 02:46:57.842957 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.842878    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 02:46:57.865174 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865153    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb902c0e-8074-4363-90c5-04eaa2c82fb3-host\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.865285 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865177    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2037171a-040e-4b0e-a635-a0b500821338-iptables-alerter-script\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.865285 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-netd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865285 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-config\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865285 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865222    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-dbus\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.865285 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865237    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjzw\" (UniqueName: \"kubernetes.io/projected/7f3cbd14-2c73-4682-97d1-9beb1df2007e-kube-api-access-lsjzw\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.865285 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865252    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-slash\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865285 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-bin\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865296    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-env-overrides\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865297    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-netd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865297    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb902c0e-8074-4363-90c5-04eaa2c82fb3-host\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865338    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-dbus\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865355    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-bin\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865376    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-host\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-tuned\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865425    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-system-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865445    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-cni-binary-copy\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865475    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-etc-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865498    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865524    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865511    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-system-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.865603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865544    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysconfig\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-sys\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-kubelet-config\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865697    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865700    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865743    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-kubelet\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865734    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865787    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25h4m\" (UniqueName: \"kubernetes.io/projected/7ffe7d41-e27f-4373-8117-c49008015a88-kube-api-access-25h4m\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865814    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-var-lib-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865851    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-env-overrides\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9913c84-a163-4a5b-b169-cc80806bc03a-ovn-node-metrics-cert\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-run\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865926    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865935    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-sys\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865938    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2037171a-040e-4b0e-a635-a0b500821338-iptables-alerter-script\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865939    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-tmp\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865641    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysconfig\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865968    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-config\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.866272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.865993    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-sys-fs\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866019    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-kubelet\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866024    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-var-lib-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866033    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-slash\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866045    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-run\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866049    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-cnibin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866070    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-sys-fs\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866069    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-cni-binary-copy\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866079    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-etc-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866119    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-etc-kubernetes\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866143    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-cnibin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866164    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-host\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866180    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-kubelet-config\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866196    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/325964e5-52a3-4672-bda2-ed01ffeee604-hosts-file\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866219    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-etc-kubernetes\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866246    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/325964e5-52a3-4672-bda2-ed01ffeee604-tmp-dir\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.867044 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866267    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/325964e5-52a3-4672-bda2-ed01ffeee604-hosts-file\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9znzz\" (UniqueName: \"kubernetes.io/projected/1c995359-9608-4568-947c-b9464cf3c4c0-kube-api-access-9znzz\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee88ccb5-4a04-431a-8963-7b8c7393604a-konnectivity-ca\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-multus\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866469    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-multus-daemon-config\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-multus-certs\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-var-lib-kubelet\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866553    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/325964e5-52a3-4672-bda2-ed01ffeee604-tmp-dir\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866650    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-var-lib-kubelet\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwzpc\" (UniqueName: \"kubernetes.io/projected/8b6ec759-f178-45cc-9c8e-7f5982c233dc-kube-api-access-vwzpc\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97v74\" (UniqueName: \"kubernetes.io/projected/fb902c0e-8074-4363-90c5-04eaa2c82fb3-kube-api-access-97v74\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866760    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866796    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-multus-certs\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866832    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee88ccb5-4a04-431a-8963-7b8c7393604a-konnectivity-ca\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866836    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-multus\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866888    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:57.868023 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866926    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866972    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-system-cni-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.866983    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867007    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-hostroot\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.867069    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867109    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-kubelet\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867140    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-multus-daemon-config\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867160    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.867269    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret podName:367d8f5b-1f4f-4d94-a9aa-7b007cd260a5 nodeName:}" failed. No retries permitted until 2026-11-02 02:46:58.36722432 +0000 UTC m=+2.310720609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret") pod "global-pull-secret-syncer-zs4kx" (UID: "367d8f5b-1f4f-4d94-a9aa-7b007cd260a5") : object "kube-system"/"original-pull-secret" not registered
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-registration-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867364    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cnibin\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867385    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867414    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-hostroot\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867388    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867388    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-bin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.868790 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867447    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cnibin\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867459    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-registration-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867480    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-system-cni-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867502    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-systemd-units\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867539    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-bin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867539    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867550    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867449    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-systemd-units\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867604    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-kubelet\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-netns\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867692    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-log-socket\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867703    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-netns\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867717    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867752    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-log-socket\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867765    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-lib-modules\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867790    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-node-log\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867829    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-systemd\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.869603 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867854    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867869    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-lib-modules\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867862    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-node-log\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee88ccb5-4a04-431a-8963-7b8c7393604a-agent-certs\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867805    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867914    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867927    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-systemd\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-netns\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.867992    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-conf-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868015    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-ovn\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868036    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-netns\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-socket-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868066    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhjgx\" (UniqueName: \"kubernetes.io/projected/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kube-api-access-mhjgx\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868089    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-conf-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868092    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtmk2\" (UniqueName: \"kubernetes.io/projected/2037171a-040e-4b0e-a635-a0b500821338-kube-api-access-dtmk2\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868121    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-socket-dir-parent\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-script-lib\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.870072 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-kubernetes\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868205    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-socket-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868220    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-ovn\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868467    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-socket-dir-parent\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868477    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-systemd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868573    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-systemd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868558    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-kubernetes\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868617    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868657    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-device-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868682    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fb902c0e-8074-4363-90c5-04eaa2c82fb3-serviceca\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.868699    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868704    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2037171a-040e-4b0e-a635-a0b500821338-host-slash\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-k8s-cni-cncf-io\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.868753    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:46:58.36873787 +0000 UTC m=+2.312234165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868771    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-k8s-cni-cncf-io\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhqj\" (UniqueName: \"kubernetes.io/projected/f9913c84-a163-4a5b-b169-cc80806bc03a-kube-api-access-9nhqj\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868803    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-modprobe-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.870674 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868820    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-device-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868841    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-conf\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868939    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-conf\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.868939    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2037171a-040e-4b0e-a635-a0b500821338-host-slash\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.869017    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-modprobe-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.869130    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg7cc\" (UniqueName: \"kubernetes.io/projected/325964e5-52a3-4672-bda2-ed01ffeee604-kube-api-access-hg7cc\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.870246    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-script-lib\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.870278    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-tmp\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.870372    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fb902c0e-8074-4363-90c5-04eaa2c82fb3-serviceca\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.870368    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.870406    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9913c84-a163-4a5b-b169-cc80806bc03a-ovn-node-metrics-cert\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.870457    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee88ccb5-4a04-431a-8963-7b8c7393604a-agent-certs\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:46:57.871123 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.870546    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-tuned\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.919826 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.919805    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 02:46:57.919826 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.919824    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 02:46:57.920001 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.919835    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h5rmj for pod openshift-network-diagnostics/network-check-target-2h58d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:46:57.920001 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:57.919895    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj podName:6090e8b3-5746-45d5-9a52-1b592b54afaf nodeName:}" failed. No retries permitted until 2026-11-02 02:46:58.419877401 +0000 UTC m=+2.363373706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h5rmj" (UniqueName: "kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj") pod "network-check-target-2h58d" (UID: "6090e8b3-5746-45d5-9a52-1b592b54afaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:46:57.922060 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.921999    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjzw\" (UniqueName: \"kubernetes.io/projected/7f3cbd14-2c73-4682-97d1-9beb1df2007e-kube-api-access-lsjzw\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:57.922278 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.922254    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25h4m\" (UniqueName: \"kubernetes.io/projected/7ffe7d41-e27f-4373-8117-c49008015a88-kube-api-access-25h4m\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:57.923175 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.923092    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwzpc\" (UniqueName: \"kubernetes.io/projected/8b6ec759-f178-45cc-9c8e-7f5982c233dc-kube-api-access-vwzpc\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:57.923283 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.923230    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9znzz\" (UniqueName: \"kubernetes.io/projected/1c995359-9608-4568-947c-b9464cf3c4c0-kube-api-access-9znzz\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:57.923520 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.923488    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97v74\" (UniqueName: \"kubernetes.io/projected/fb902c0e-8074-4363-90c5-04eaa2c82fb3-kube-api-access-97v74\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:57.923829 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.923813    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhjgx\" (UniqueName: \"kubernetes.io/projected/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kube-api-access-mhjgx\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:57.924135 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.924118    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtmk2\" (UniqueName: \"kubernetes.io/projected/2037171a-040e-4b0e-a635-a0b500821338-kube-api-access-dtmk2\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:57.924669 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.924650    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhqj\" (UniqueName: \"kubernetes.io/projected/f9913c84-a163-4a5b-b169-cc80806bc03a-kube-api-access-9nhqj\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:57.925216 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:57.925201    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg7cc\" (UniqueName: \"kubernetes.io/projected/325964e5-52a3-4672-bda2-ed01ffeee604-kube-api-access-hg7cc\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:58.011580 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.011549    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:46:58.028244 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.028218    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:46:58.066906 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.066876    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:46:58.091393 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.091370    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bvqdp"
Nov 02 02:46:58.092430 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.092416    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:46:58.111987 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.111964    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:46:58.116337 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.116324    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:46:58.133113 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.133096    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:46:58.135841 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.135811    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fdb79d33d08466df57752003cd13435.slice/crio-b7ee88a74b0b26072c9fcd0534e1ee22a726f6758c817131152357117a1bd974 WatchSource:0}: Error finding container b7ee88a74b0b26072c9fcd0534e1ee22a726f6758c817131152357117a1bd974: Status 404 returned error can't find the container with id b7ee88a74b0b26072c9fcd0534e1ee22a726f6758c817131152357117a1bd974
Nov 02 02:46:58.136354 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.136328    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9913c84_a163_4a5b_b169_cc80806bc03a.slice/crio-17150c480564a5c8773c9aa24ad3f72ca311899ce3fa8b24a23daa341fa2472d WatchSource:0}: Error finding container 17150c480564a5c8773c9aa24ad3f72ca311899ce3fa8b24a23daa341fa2472d: Status 404 returned error can't find the container with id 17150c480564a5c8773c9aa24ad3f72ca311899ce3fa8b24a23daa341fa2472d
Nov 02 02:46:58.137275 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.137252    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832ceb98d0ffc61a89e76de2f575c0c6.slice/crio-83b2570bb7df51d94ef3cb804af0ded34fdcd2c845439b622bb3ab10c0f5f369 WatchSource:0}: Error finding container 83b2570bb7df51d94ef3cb804af0ded34fdcd2c845439b622bb3ab10c0f5f369: Status 404 returned error can't find the container with id 83b2570bb7df51d94ef3cb804af0ded34fdcd2c845439b622bb3ab10c0f5f369
Nov 02 02:46:58.137986 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.137958    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6ec759_f178_45cc_9c8e_7f5982c233dc.slice/crio-592c448658b7a9114bce6609f6dba5aeca862d34ead287a7208dc5f3f7b4cf66 WatchSource:0}: Error finding container 592c448658b7a9114bce6609f6dba5aeca862d34ead287a7208dc5f3f7b4cf66: Status 404 returned error can't find the container with id 592c448658b7a9114bce6609f6dba5aeca862d34ead287a7208dc5f3f7b4cf66
Nov 02 02:46:58.142778 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.142761    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:46:58.143344 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.143231    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee88ccb5_4a04_431a_8963_7b8c7393604a.slice/crio-594328116a7108957d4adb7600c006728ed8f54dae2fb0520293b1e26ca7b1db WatchSource:0}: Error finding container 594328116a7108957d4adb7600c006728ed8f54dae2fb0520293b1e26ca7b1db: Status 404 returned error can't find the container with id 594328116a7108957d4adb7600c006728ed8f54dae2fb0520293b1e26ca7b1db
Nov 02 02:46:58.143432 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.143374    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 02:46:58.151963 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.151943    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c73be0_a65c_490e_9c4e_e4c15006f5c8.slice/crio-01f0668a0cc86cc92260ac27d61b4909d23f7d9988a28ead3c4a7206ed01f779 WatchSource:0}: Error finding container 01f0668a0cc86cc92260ac27d61b4909d23f7d9988a28ead3c4a7206ed01f779: Status 404 returned error can't find the container with id 01f0668a0cc86cc92260ac27d61b4909d23f7d9988a28ead3c4a7206ed01f779
Nov 02 02:46:58.287616 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.287588    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:46:58.372234 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.372205    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:58.372402 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.372242    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:58.372402 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:58.372329    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:46:58.372402 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:58.372342    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 02:46:58.372402 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:58.372384    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:46:59.372365341 +0000 UTC m=+3.315861634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:46:58.372540 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:58.372404    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret podName:367d8f5b-1f4f-4d94-a9aa-7b007cd260a5 nodeName:}" failed. No retries permitted until 2026-11-02 02:46:59.372391226 +0000 UTC m=+3.315887519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret") pod "global-pull-secret-syncer-zs4kx" (UID: "367d8f5b-1f4f-4d94-a9aa-7b007cd260a5") : object "kube-system"/"original-pull-secret" not registered
Nov 02 02:46:58.471027 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.470992    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:46:58.473401 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.473379    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:46:58.473591 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:58.473574    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 02:46:58.473689 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:58.473601    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 02:46:58.473689 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:58.473614    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h5rmj for pod openshift-network-diagnostics/network-check-target-2h58d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:46:58.473805 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:58.473703    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj podName:6090e8b3-5746-45d5-9a52-1b592b54afaf nodeName:}" failed. No retries permitted until 2026-11-02 02:46:59.473679725 +0000 UTC m=+3.417176029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5rmj" (UniqueName: "kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj") pod "network-check-target-2h58d" (UID: "6090e8b3-5746-45d5-9a52-1b592b54afaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:46:58.539328 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.539277    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"9fdb79d33d08466df57752003cd13435","Type":"ContainerStarted","Data":"b7ee88a74b0b26072c9fcd0534e1ee22a726f6758c817131152357117a1bd974"}
Nov 02 02:46:58.541342 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.541299    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd" event={"ID":"77c73be0-a65c-490e-9c4e-e4c15006f5c8","Type":"ContainerStarted","Data":"01f0668a0cc86cc92260ac27d61b4909d23f7d9988a28ead3c4a7206ed01f779"}
Nov 02 02:46:58.542403 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.542383    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b5wvk" event={"ID":"ee88ccb5-4a04-431a-8963-7b8c7393604a","Type":"ContainerStarted","Data":"594328116a7108957d4adb7600c006728ed8f54dae2fb0520293b1e26ca7b1db"}
Nov 02 02:46:58.543481 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.543461    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q8k8l" event={"ID":"8b6ec759-f178-45cc-9c8e-7f5982c233dc","Type":"ContainerStarted","Data":"592c448658b7a9114bce6609f6dba5aeca862d34ead287a7208dc5f3f7b4cf66"}
Nov 02 02:46:58.544633 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.544597    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"832ceb98d0ffc61a89e76de2f575c0c6","Type":"ContainerStarted","Data":"83b2570bb7df51d94ef3cb804af0ded34fdcd2c845439b622bb3ab10c0f5f369"}
Nov 02 02:46:58.545804 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.545786    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"17150c480564a5c8773c9aa24ad3f72ca311899ce3fa8b24a23daa341fa2472d"}
Nov 02 02:46:58.659062 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.659028    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:46:58.803100 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.802990    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 02:41:57 +0000 UTC" deadline="2028-07-23 13:49:46.252217248 +0000 UTC"
Nov 02 02:46:58.803100 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:58.803042    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15107h2m47.449178872s"
Nov 02 02:46:58.955923 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.955881    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb902c0e_8074_4363_90c5_04eaa2c82fb3.slice/crio-81ae5c591b76617400b6e04b4fa6070fcd4ec7daf6815b4a4fe63f04b5f05e3b WatchSource:0}: Error finding container 81ae5c591b76617400b6e04b4fa6070fcd4ec7daf6815b4a4fe63f04b5f05e3b: Status 404 returned error can't find the container with id 81ae5c591b76617400b6e04b4fa6070fcd4ec7daf6815b4a4fe63f04b5f05e3b
Nov 02 02:46:58.963333 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.963300    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325964e5_52a3_4672_bda2_ed01ffeee604.slice/crio-d2ee2156b68aacc4ae5745a6bd6b5f3a14c67984fa4649f701236156ae8df082 WatchSource:0}: Error finding container d2ee2156b68aacc4ae5745a6bd6b5f3a14c67984fa4649f701236156ae8df082: Status 404 returned error can't find the container with id d2ee2156b68aacc4ae5745a6bd6b5f3a14c67984fa4649f701236156ae8df082
Nov 02 02:46:58.966497 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.966475    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ffe7d41_e27f_4373_8117_c49008015a88.slice/crio-802e7d2db4353e199fb67d5d3ac0334114278aa867ed8a0b4f9bd78a1543d8cf WatchSource:0}: Error finding container 802e7d2db4353e199fb67d5d3ac0334114278aa867ed8a0b4f9bd78a1543d8cf: Status 404 returned error can't find the container with id 802e7d2db4353e199fb67d5d3ac0334114278aa867ed8a0b4f9bd78a1543d8cf
Nov 02 02:46:58.969825 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:58.969801    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2037171a_040e_4b0e_a635_a0b500821338.slice/crio-e638179b7cdf78468e34d16ac6f6ac761dc24a4c8dba543523f47936fe050e02 WatchSource:0}: Error finding container e638179b7cdf78468e34d16ac6f6ac761dc24a4c8dba543523f47936fe050e02: Status 404 returned error can't find the container with id e638179b7cdf78468e34d16ac6f6ac761dc24a4c8dba543523f47936fe050e02
Nov 02 02:46:59.010747 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:46:59.010724    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3cbd14_2c73_4682_97d1_9beb1df2007e.slice/crio-2aaef35ea454502d01836b9e66ed47e020027f675e56df6e84fe667043f80b84 WatchSource:0}: Error finding container 2aaef35ea454502d01836b9e66ed47e020027f675e56df6e84fe667043f80b84: Status 404 returned error can't find the container with id 2aaef35ea454502d01836b9e66ed47e020027f675e56df6e84fe667043f80b84
Nov 02 02:46:59.382377 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.382339    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:59.382575 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.382404    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:59.382575 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.382535    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:46:59.382738 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.382600    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:01.382580788 +0000 UTC m=+5.326077093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:46:59.382738 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.382704    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 02:46:59.382873 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.382740    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret podName:367d8f5b-1f4f-4d94-a9aa-7b007cd260a5 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:01.382728153 +0000 UTC m=+5.326224457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret") pod "global-pull-secret-syncer-zs4kx" (UID: "367d8f5b-1f4f-4d94-a9aa-7b007cd260a5") : object "kube-system"/"original-pull-secret" not registered
Nov 02 02:46:59.483217 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.483175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:46:59.483744 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.483360    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 02:46:59.483744 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.483385    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 02:46:59.483744 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.483399    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h5rmj for pod openshift-network-diagnostics/network-check-target-2h58d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:46:59.483744 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.483469    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj podName:6090e8b3-5746-45d5-9a52-1b592b54afaf nodeName:}" failed. No retries permitted until 2026-11-02 02:47:01.483450279 +0000 UTC m=+5.426946587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5rmj" (UniqueName: "kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj") pod "network-check-target-2h58d" (UID: "6090e8b3-5746-45d5-9a52-1b592b54afaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:46:59.536020 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.535986    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:46:59.536206 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.536132    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:46:59.536277 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.536218    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:46:59.536328 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.536292    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:46:59.536393 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.536360    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:46:59.536462 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:46:59.536442    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:46:59.550526 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.550493    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fbwhk" event={"ID":"2037171a-040e-4b0e-a635-a0b500821338","Type":"ContainerStarted","Data":"e638179b7cdf78468e34d16ac6f6ac761dc24a4c8dba543523f47936fe050e02"}
Nov 02 02:46:59.556357 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.556323    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" event={"ID":"7f3cbd14-2c73-4682-97d1-9beb1df2007e","Type":"ContainerStarted","Data":"2aaef35ea454502d01836b9e66ed47e020027f675e56df6e84fe667043f80b84"}
Nov 02 02:46:59.558530 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.558495    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bvqdp" event={"ID":"7ffe7d41-e27f-4373-8117-c49008015a88","Type":"ContainerStarted","Data":"802e7d2db4353e199fb67d5d3ac0334114278aa867ed8a0b4f9bd78a1543d8cf"}
Nov 02 02:46:59.561101 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.561072    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z7p4b" event={"ID":"325964e5-52a3-4672-bda2-ed01ffeee604","Type":"ContainerStarted","Data":"d2ee2156b68aacc4ae5745a6bd6b5f3a14c67984fa4649f701236156ae8df082"}
Nov 02 02:46:59.563501 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.563475    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mdddz" event={"ID":"fb902c0e-8074-4363-90c5-04eaa2c82fb3","Type":"ContainerStarted","Data":"81ae5c591b76617400b6e04b4fa6070fcd4ec7daf6815b4a4fe63f04b5f05e3b"}
Nov 02 02:46:59.804310 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.804253    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 02:41:57 +0000 UTC" deadline="2028-04-12 19:11:52.75267752 +0000 UTC"
Nov 02 02:46:59.804310 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:46:59.804289    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12664h24m52.948392894s"
Nov 02 02:47:01.402033 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:01.401995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:01.402664 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:01.402051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:01.402664 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.402178    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:47:01.402664 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.402233    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:05.40222014 +0000 UTC m=+9.345716444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:47:01.402664 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.402286    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 02:47:01.402664 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.402310    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret podName:367d8f5b-1f4f-4d94-a9aa-7b007cd260a5 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:05.402302674 +0000 UTC m=+9.345798979 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret") pod "global-pull-secret-syncer-zs4kx" (UID: "367d8f5b-1f4f-4d94-a9aa-7b007cd260a5") : object "kube-system"/"original-pull-secret" not registered
Nov 02 02:47:01.503113 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:01.503072    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:01.503313 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.503295    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 02:47:01.503379 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.503325    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 02:47:01.503379 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.503340    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h5rmj for pod openshift-network-diagnostics/network-check-target-2h58d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:47:01.503497 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.503402    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj podName:6090e8b3-5746-45d5-9a52-1b592b54afaf nodeName:}" failed. No retries permitted until 2026-11-02 02:47:05.503383988 +0000 UTC m=+9.446880297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5rmj" (UniqueName: "kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj") pod "network-check-target-2h58d" (UID: "6090e8b3-5746-45d5-9a52-1b592b54afaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:47:01.536320 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:01.536286    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:01.536491 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.536408    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:01.536491 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:01.536469    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:01.536666 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.536519    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:01.536666 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:01.536554    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:01.536666 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:01.536594    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:03.536087 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.535860    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:03.536488 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.536098    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:03.536488 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:03.536179    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:03.536488 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:03.536257    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:03.536488 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.536299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:03.536488 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:03.536379    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:03.604792 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.604311    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="b39d9b2fddde2f5ad3c2090481d85895fa21009b88b968042013a4101b427e22" exitCode=0
Nov 02 02:47:03.604792 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.604409    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" event={"ID":"7f3cbd14-2c73-4682-97d1-9beb1df2007e","Type":"ContainerDied","Data":"b39d9b2fddde2f5ad3c2090481d85895fa21009b88b968042013a4101b427e22"}
Nov 02 02:47:03.613211 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.612745    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"9fdb79d33d08466df57752003cd13435","Type":"ContainerStarted","Data":"8bd8367fad8df8e5525b96f5d5ff6dcfebf53ee84c344ff759dd1ab4491fd1fc"}
Nov 02 02:47:03.624724 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.624668    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b5wvk" event={"ID":"ee88ccb5-4a04-431a-8963-7b8c7393604a","Type":"ContainerStarted","Data":"9a88bd58dcdeb157a198a788f633290d652fb1a23fa73c7af4bf7e073b954efa"}
Nov 02 02:47:03.631553 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.631009    2574 generic.go:356] "Generic (PLEG): container finished" podID="832ceb98d0ffc61a89e76de2f575c0c6" containerID="d36bbbe18bf3b260026bdf406306a60ca3cfdf4737333be6f4102c7396f2789b" exitCode=0
Nov 02 02:47:03.631553 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:03.631048    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"832ceb98d0ffc61a89e76de2f575c0c6","Type":"ContainerDied","Data":"d36bbbe18bf3b260026bdf406306a60ca3cfdf4737333be6f4102c7396f2789b"}
Nov 02 02:47:04.055877 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:04.054992    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b5wvk" podStartSLOduration=1.8983948499999999 podStartE2EDuration="7.054971814s" totalImagesPullingTime="5.156576964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:03.945087994 +0000 UTC m=+7.888584297" watchObservedRunningTime="2026-11-02 02:47:04.054971814 +0000 UTC m=+7.998468126"
Nov 02 02:47:04.167358 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:04.166795    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal" podStartSLOduration=7.166774068 podStartE2EDuration="7.166774068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:04.055721675 +0000 UTC m=+7.999217989" watchObservedRunningTime="2026-11-02 02:47:04.166774068 +0000 UTC m=+8.110270378"
Nov 02 02:47:04.364639 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:04.364477    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:04.365141 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:04.365119    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:04.651015 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:04.650930    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"832ceb98d0ffc61a89e76de2f575c0c6","Type":"ContainerStarted","Data":"5f5dbcfc65a07fbe0177081ae6c217fd2d6d0a5d1cf99322ef2e4ca1a8392a14"}
Nov 02 02:47:05.440158 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.440082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:05.440380 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.440170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:05.440380 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.440310    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 02:47:05.440380 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.440367    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret podName:367d8f5b-1f4f-4d94-a9aa-7b007cd260a5 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:13.440349179 +0000 UTC m=+17.383845469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret") pod "global-pull-secret-syncer-zs4kx" (UID: "367d8f5b-1f4f-4d94-a9aa-7b007cd260a5") : object "kube-system"/"original-pull-secret" not registered
Nov 02 02:47:05.440613 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.440428    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:47:05.440613 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.440460    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:13.440448777 +0000 UTC m=+17.383945068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:47:05.535684 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.535654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:05.535862 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.535783    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:05.535934 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.535869    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:05.535986 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.535950    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:05.536031 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.536000    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:05.536083 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.536067    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:05.541531 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.541503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:05.541689 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.541675    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 02:47:05.541754 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.541694    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 02:47:05.541754 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.541707    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h5rmj for pod openshift-network-diagnostics/network-check-target-2h58d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:47:05.541869 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:05.541760    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj podName:6090e8b3-5746-45d5-9a52-1b592b54afaf nodeName:}" failed. No retries permitted until 2026-11-02 02:47:13.541741497 +0000 UTC m=+17.485237791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5rmj" (UniqueName: "kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj") pod "network-check-target-2h58d" (UID: "6090e8b3-5746-45d5-9a52-1b592b54afaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:47:05.656101 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.656063    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mdddz" event={"ID":"fb902c0e-8074-4363-90c5-04eaa2c82fb3","Type":"ContainerStarted","Data":"d393ca5ba4ef173dc2e20ab6b9bcf73f26c31b7a416428fc383fd666fff0cb5a"}
Nov 02 02:47:05.659256 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.659227    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd" event={"ID":"77c73be0-a65c-490e-9c4e-e4c15006f5c8","Type":"ContainerStarted","Data":"650cfdd701dbb7ef143c62e98d3524fd1c7a7206d63a9dec8658e508da2704b9"}
Nov 02 02:47:05.659389 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.659321    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 02:47:05.768793 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:05.768686    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal" podStartSLOduration=8.768664505 podStartE2EDuration="8.768664505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:04.763608751 +0000 UTC m=+8.707105062" watchObservedRunningTime="2026-11-02 02:47:05.768664505 +0000 UTC m=+9.712160816"
Nov 02 02:47:07.196421 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.196151    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 02:47:07.536306 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.536268    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:07.536492 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:07.536389    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:07.536492 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.536477    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:07.536612 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:07.536560    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:07.536703 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.536614    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:07.536773 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:07.536721    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:07.540431 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.539894    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T02:47:07.196175643Z","UUID":"a8095609-4a57-4b9c-8ccb-d0c95bc4c955","Handler":null,"Name":"","Endpoint":""}
Nov 02 02:47:07.541562 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.541540    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 02:47:07.541680 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.541573    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 02:47:07.666734 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.666672    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bvqdp" event={"ID":"7ffe7d41-e27f-4373-8117-c49008015a88","Type":"ContainerStarted","Data":"c344e66d4293224a223215cc2df624341b8160c1a2a1f35a417ec75844b26e66"}
Nov 02 02:47:07.669659 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.669616    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd" event={"ID":"77c73be0-a65c-490e-9c4e-e4c15006f5c8","Type":"ContainerStarted","Data":"62f303c32cba5c5024dd1dae5c4de47b56d1f8914efc5b032898ca1ec533a363"}
Nov 02 02:47:07.785600 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:07.785541    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mdddz" podStartSLOduration=4.881446764 podStartE2EDuration="10.785522884s" totalImagesPullingTime="5.90407612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:05.769122841 +0000 UTC m=+9.712619151" watchObservedRunningTime="2026-11-02 02:47:07.785522884 +0000 UTC m=+11.729019195"
Nov 02 02:47:08.219290 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:08.219209    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:08.219798 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:08.219400    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 02:47:08.219980 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:08.219874    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:08.331516 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:08.331462    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-bvqdp" podStartSLOduration=2.9525270839999997 podStartE2EDuration="11.331441182s" totalImagesPullingTime="8.378914098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:07.784841767 +0000 UTC m=+11.728338078" watchObservedRunningTime="2026-11-02 02:47:08.331441182 +0000 UTC m=+12.274937492"
Nov 02 02:47:08.676879 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:08.676838    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="41438c9ad61ea3e11c231d542aa0d17022721a0010da6c6380db268e48ada882" exitCode=0
Nov 02 02:47:08.677048 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:08.676879    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" event={"ID":"7f3cbd14-2c73-4682-97d1-9beb1df2007e","Type":"ContainerDied","Data":"41438c9ad61ea3e11c231d542aa0d17022721a0010da6c6380db268e48ada882"}
Nov 02 02:47:08.679958 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:08.679934    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z7p4b" event={"ID":"325964e5-52a3-4672-bda2-ed01ffeee604","Type":"ContainerStarted","Data":"f578f8975e0c640163e826815bba66f1ec2329c7a511d8baed064161fe1d9e8c"}
Nov 02 02:47:09.536265 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:09.536240    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:09.536747 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:09.536268    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:09.536747 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:09.536334    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:09.536747 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:09.536380    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:09.536747 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:09.536494    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:09.536747 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:09.536580    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:09.686660 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:09.686400    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd" event={"ID":"77c73be0-a65c-490e-9c4e-e4c15006f5c8","Type":"ContainerStarted","Data":"137e2b0bd433c296e189157ffcc9dcc4b853da657991991f34741ef93de5c452"}
Nov 02 02:47:09.688474 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:09.688452    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fbwhk" event={"ID":"2037171a-040e-4b0e-a635-a0b500821338","Type":"ContainerStarted","Data":"0735bb63eadfb8fe9de39c3b932efadf0641b12c5704f2a8f049be7e4ed02442"}
Nov 02 02:47:09.809302 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:09.809193    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd" podStartSLOduration=2.414657707 podStartE2EDuration="12.809177695s" totalImagesPullingTime="10.394519988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:09.808770217 +0000 UTC m=+13.752266528" watchObservedRunningTime="2026-11-02 02:47:09.809177695 +0000 UTC m=+13.752674004"
Nov 02 02:47:09.809445 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:09.809321    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-z7p4b" podStartSLOduration=3.974155182 podStartE2EDuration="12.80931629s" totalImagesPullingTime="8.835161108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:08.902820707 +0000 UTC m=+12.846317016" watchObservedRunningTime="2026-11-02 02:47:09.80931629 +0000 UTC m=+13.752812600"
Nov 02 02:47:09.918966 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:09.918914    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fbwhk" podStartSLOduration=4.296491047 podStartE2EDuration="12.918899175s" totalImagesPullingTime="8.622408128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:09.918436861 +0000 UTC m=+13.861933172" watchObservedRunningTime="2026-11-02 02:47:09.918899175 +0000 UTC m=+13.862395484"
Nov 02 02:47:10.692387 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:10.692355    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q8k8l" event={"ID":"8b6ec759-f178-45cc-9c8e-7f5982c233dc","Type":"ContainerStarted","Data":"eb76d843033e2d9538f10ec7dab7e70f5fa8f2699cf90bbe320ba6d2c7cc0efa"}
Nov 02 02:47:10.694895 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:10.694864    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="2250986d8fc2b6696dac9a5629bf9de3e495bab1b950440495cd0984fde6ab71" exitCode=0
Nov 02 02:47:10.695016 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:10.694948    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" event={"ID":"7f3cbd14-2c73-4682-97d1-9beb1df2007e","Type":"ContainerDied","Data":"2250986d8fc2b6696dac9a5629bf9de3e495bab1b950440495cd0984fde6ab71"}
Nov 02 02:47:10.805341 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:10.805286    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q8k8l" podStartSLOduration=2.309050042 podStartE2EDuration="13.805269378s" totalImagesPullingTime="11.496219336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:10.804924769 +0000 UTC m=+14.748421080" watchObservedRunningTime="2026-11-02 02:47:10.805269378 +0000 UTC m=+14.748765688"
Nov 02 02:47:11.536272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:11.536232    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:11.536272 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:11.536257    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:11.536511 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:11.536365    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:11.536511 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:11.536369    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:11.536511 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:11.536443    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:11.536684 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:11.536518    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:12.702386 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:12.702349    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="c69ef9f55f8d48b9301431c292bdc657f836b2df47af2b032c714c1ae4defa8a" exitCode=0
Nov 02 02:47:12.703206 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:12.702404    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" event={"ID":"7f3cbd14-2c73-4682-97d1-9beb1df2007e","Type":"ContainerDied","Data":"c69ef9f55f8d48b9301431c292bdc657f836b2df47af2b032c714c1ae4defa8a"}
Nov 02 02:47:13.505830 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:13.505799    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:13.505991 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:13.505858    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:13.505991 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.505948    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:47:13.506057 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.506003    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:29.505989079 +0000 UTC m=+33.449485380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 02:47:13.506057 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.505948    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 02:47:13.506121 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.506067    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret podName:367d8f5b-1f4f-4d94-a9aa-7b007cd260a5 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:29.506056365 +0000 UTC m=+33.449552670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret") pod "global-pull-secret-syncer-zs4kx" (UID: "367d8f5b-1f4f-4d94-a9aa-7b007cd260a5") : object "kube-system"/"original-pull-secret" not registered
Nov 02 02:47:13.535914 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:13.535892    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:13.535985 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:13.535914    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:13.535985 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:13.535893    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:13.536048 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.535982    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:13.536081 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.536065    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:13.536122 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.536106    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:13.606282 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:13.606249    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:13.606416 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.606367    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 02:47:13.606416 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.606382    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 02:47:13.606416 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.606392    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h5rmj for pod openshift-network-diagnostics/network-check-target-2h58d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:47:13.606503 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:13.606439    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj podName:6090e8b3-5746-45d5-9a52-1b592b54afaf nodeName:}" failed. No retries permitted until 2026-11-02 02:47:29.606426591 +0000 UTC m=+33.549922879 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5rmj" (UniqueName: "kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj") pod "network-check-target-2h58d" (UID: "6090e8b3-5746-45d5-9a52-1b592b54afaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 02:47:14.710255 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:14.710064    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"78b1c1c08884e8e2ac1861d432016f15432ca303dfe5ca365ef1931b9bc287e3"}
Nov 02 02:47:14.710724 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:14.710308    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"ca69a9cb2e8375f20be3d74077b0e4c3bb7b1fd64f39831c262bdd4dd806d140"}
Nov 02 02:47:14.710724 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:14.710349    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"2fb177777078137f60573c7f20b0e8ecbd4e68bd5d3f359b939e619f071ed79f"}
Nov 02 02:47:14.710724 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:14.710377    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"9524575dec0cfaf19cfa1d555540af5df19697256f8e789168f40ae71ddcda27"}
Nov 02 02:47:14.710724 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:14.710415    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"99e24cc36691d72349811376cb10f185b046ff99f059a86bb2e450b9e85a4356"}
Nov 02 02:47:15.535921 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:15.535727    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:15.536107 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:15.535754    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:15.536107 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:15.536001    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:15.536107 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:15.535797    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:15.536107 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:15.536065    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:15.536293 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:15.536139    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:15.714998 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:15.714964    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"9bf7237016c04d14196b9e255276f85bad9b2290db656cfb10d84900d7c1d49b"}
Nov 02 02:47:15.717574 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:15.717552    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="78b74a5e46d6e4f48029877a0e935094a2545b80461f3e1da94e628b6e990ebd" exitCode=0
Nov 02 02:47:15.717711 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:15.717581    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" event={"ID":"7f3cbd14-2c73-4682-97d1-9beb1df2007e","Type":"ContainerDied","Data":"78b74a5e46d6e4f48029877a0e935094a2545b80461f3e1da94e628b6e990ebd"}
Nov 02 02:47:16.722609 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:16.722570    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="09f67c1e5d2274e6d58ce21752f8afd23b1648deb46d893ee0760001bfa6b6e8" exitCode=0
Nov 02 02:47:16.722609 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:16.722610    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" event={"ID":"7f3cbd14-2c73-4682-97d1-9beb1df2007e","Type":"ContainerDied","Data":"09f67c1e5d2274e6d58ce21752f8afd23b1648deb46d893ee0760001bfa6b6e8"}
Nov 02 02:47:17.535600 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:17.535563    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:17.535600 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:17.535581    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:17.535831 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:17.535685    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:17.535831 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:17.535689    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:17.535831 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:17.535753    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:17.536024 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:17.535822    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:17.727760 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:17.727721    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"af1e88213ce6d1adb0767e50a69ceff03e5913ab96152fd47f199d8c120c7468"}
Nov 02 02:47:17.730744 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:17.730721    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" event={"ID":"7f3cbd14-2c73-4682-97d1-9beb1df2007e","Type":"ContainerStarted","Data":"4ddb2dda987bd74042533b865c5597e7cc2d6855e96f44489660ea26319d5283"}
Nov 02 02:47:17.849162 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:17.849071    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2rm9b" podStartSLOduration=7.857364023 podStartE2EDuration="20.849057823s" totalImagesPullingTime="12.9916938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:17.848845756 +0000 UTC m=+21.792342067" watchObservedRunningTime="2026-11-02 02:47:17.849057823 +0000 UTC m=+21.792554133"
Nov 02 02:47:19.535999 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:19.535967    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:19.535999 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:19.535982    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:19.536757 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:19.536075    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:19.536757 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:19.536071    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:19.536757 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:19.536132    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:19.536757 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:19.536249    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:19.737453 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:19.737419    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" event={"ID":"f9913c84-a163-4a5b-b169-cc80806bc03a","Type":"ContainerStarted","Data":"4c23f6d5709676d66523f57e5fd5c1f9ba04f1e9f4f220ff6c93e3a1ace53855"}
Nov 02 02:47:19.737677 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:19.737661    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:19.753812 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:19.753790    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:20.002891 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:20.002850    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv" podStartSLOduration=6.772975977 podStartE2EDuration="23.002836231s" totalImagesPullingTime="16.229860254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:19.953579988 +0000 UTC m=+23.897076298" watchObservedRunningTime="2026-11-02 02:47:20.002836231 +0000 UTC m=+23.946332537"
Nov 02 02:47:20.036077 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:20.036044    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:20.740005 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:20.739974    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:20.755354 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:20.755332    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:21.503547 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:21.503373    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zs4kx"]
Nov 02 02:47:21.503715 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:21.503652    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:21.503767 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:21.503736    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:21.506491 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:21.506465    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2h58d"]
Nov 02 02:47:21.506617 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:21.506592    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:21.506731 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:21.506711    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:21.507134 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:21.507109    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4xrlc"]
Nov 02 02:47:21.507253 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:21.507195    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:21.507318 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:21.507283    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:23.535930 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:23.535896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:23.535930 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:23.535927    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:23.536452 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:23.535901    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:23.536452 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:23.536025    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:23.536452 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:23.536136    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:23.536452 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:23.536210    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:25.535968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:25.535938    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:25.535968 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:25.535968    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:25.536413 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:25.536030    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:25.536413 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:25.536041    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xrlc" podUID="1c995359-9608-4568-947c-b9464cf3c4c0"
Nov 02 02:47:25.536413 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:25.536099    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zs4kx" podUID="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5"
Nov 02 02:47:25.536413 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:25.536187    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h58d" podUID="6090e8b3-5746-45d5-9a52-1b592b54afaf"
Nov 02 02:47:27.348326 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.348299    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeReady"
Nov 02 02:47:27.348730 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.348428    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 02 02:47:27.361771 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.361739    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:27.501058 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.501018    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-st5ft"]
Nov 02 02:47:27.504342 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.504315    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n4jz2"]
Nov 02 02:47:27.504506 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.504466    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.507243 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.507223    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-st5ft"]
Nov 02 02:47:27.507348 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.507331    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:27.510243 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.510223    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n4jz2"]
Nov 02 02:47:27.535374 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.535337    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:27.535374 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.535373    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:27.535515 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.535486    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:27.556252 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.556228    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 02:47:27.556451 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.556424    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 02:47:27.556578 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.556563    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gdqg4\""
Nov 02 02:47:27.559167 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.559154    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 02:47:27.594726 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.594702    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:27.598733 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.598697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.598804 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.598743    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b9f52ac-c454-4aa5-bb64-3bdae890695d-config-volume\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.598804 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.598759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b9f52ac-c454-4aa5-bb64-3bdae890695d-tmp-dir\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.598877 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.598858    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6grpn\" (UniqueName: \"kubernetes.io/projected/3b9f52ac-c454-4aa5-bb64-3bdae890695d-kube-api-access-6grpn\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.598909 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.598895    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/149f4882-f029-488f-ba54-99d60c431352-cert\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:27.598940 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.598916    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9v8q\" (UniqueName: \"kubernetes.io/projected/149f4882-f029-488f-ba54-99d60c431352-kube-api-access-w9v8q\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:27.657759 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657725    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 02:47:27.657759 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657725    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 02:47:27.657759 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657754    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 02:47:27.658063 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657734    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 02:47:27.658063 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657783    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-wsjnf\""
Nov 02 02:47:27.658063 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657910    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 02:47:27.658063 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657947    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6md5s\""
Nov 02 02:47:27.658063 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657963    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9gkth\""
Nov 02 02:47:27.658063 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.657965    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 02:47:27.658063 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.658021    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x92pp\""
Nov 02 02:47:27.699658 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.699610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9v8q\" (UniqueName: \"kubernetes.io/projected/149f4882-f029-488f-ba54-99d60c431352-kube-api-access-w9v8q\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:27.699814 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.699667    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.699814 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.699734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b9f52ac-c454-4aa5-bb64-3bdae890695d-config-volume\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.699814 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.699758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b9f52ac-c454-4aa5-bb64-3bdae890695d-tmp-dir\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.699814 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.699800    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6grpn\" (UniqueName: \"kubernetes.io/projected/3b9f52ac-c454-4aa5-bb64-3bdae890695d-kube-api-access-6grpn\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.700019 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:27.699857    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:27.700019 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:27.699915    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:28.199896744 +0000 UTC m=+32.143393034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:27.700019 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.699945    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/149f4882-f029-488f-ba54-99d60c431352-cert\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:27.700210 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.700180    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b9f52ac-c454-4aa5-bb64-3bdae890695d-tmp-dir\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.700441 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.700423    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b9f52ac-c454-4aa5-bb64-3bdae890695d-config-volume\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.703637 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.703607    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/149f4882-f029-488f-ba54-99d60c431352-cert\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:27.756734 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.756703    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9v8q\" (UniqueName: \"kubernetes.io/projected/149f4882-f029-488f-ba54-99d60c431352-kube-api-access-w9v8q\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:27.756734 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.756727    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6grpn\" (UniqueName: \"kubernetes.io/projected/3b9f52ac-c454-4aa5-bb64-3bdae890695d-kube-api-access-6grpn\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:27.822811 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:27.822782    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:28.198443 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:28.198371    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n4jz2"]
Nov 02 02:47:28.201738 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:47:28.201713    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod149f4882_f029_488f_ba54_99d60c431352.slice/crio-00980703f376325d582c4f2157a928fdd9f8b8657cecbff59e46352d5e71daa5 WatchSource:0}: Error finding container 00980703f376325d582c4f2157a928fdd9f8b8657cecbff59e46352d5e71daa5: Status 404 returned error can't find the container with id 00980703f376325d582c4f2157a928fdd9f8b8657cecbff59e46352d5e71daa5
Nov 02 02:47:28.204126 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:28.204101    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:28.204295 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:28.204250    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:28.204328 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:28.204308    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:29.204293722 +0000 UTC m=+33.147790020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:28.758021 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:28.757985    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n4jz2" event={"ID":"149f4882-f029-488f-ba54-99d60c431352","Type":"ContainerStarted","Data":"00980703f376325d582c4f2157a928fdd9f8b8657cecbff59e46352d5e71daa5"}
Nov 02 02:47:29.212757 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:29.212682    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:29.212899 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:29.212834    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:29.212899 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:29.212897    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:31.212883017 +0000 UTC m=+35.156379306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:29.515217 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:29.515187    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:29.515409 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:29.515243    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:29.515409 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:29.515384    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 02:47:29.515521 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:29.515456    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:48:01.515437047 +0000 UTC m=+65.458933351 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : secret "metrics-daemon-secret" not found
Nov 02 02:47:29.517922 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:29.517896    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:29.615714 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:29.615678    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:29.618821 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:29.618796    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:29.651663 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:29.651615    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:29.657078 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:29.657061    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:30.026880 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:30.026852    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2h58d"]
Nov 02 02:47:30.030911 ip-10-0-131-112 kubenswrapper[2574]: W1102 02:47:30.030884    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6090e8b3_5746_45d5_9a52_1b592b54afaf.slice/crio-ff186b20a78179e47a590d786c2cd5a59a325054f36d187dfcabce39e9be48b7 WatchSource:0}: Error finding container ff186b20a78179e47a590d786c2cd5a59a325054f36d187dfcabce39e9be48b7: Status 404 returned error can't find the container with id ff186b20a78179e47a590d786c2cd5a59a325054f36d187dfcabce39e9be48b7
Nov 02 02:47:30.076966 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:30.076937    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zs4kx"]
Nov 02 02:47:30.552169 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:30.552135    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:30.698885 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:30.698852    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:30.764388 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:30.764356    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2h58d" event={"ID":"6090e8b3-5746-45d5-9a52-1b592b54afaf","Type":"ContainerStarted","Data":"ff186b20a78179e47a590d786c2cd5a59a325054f36d187dfcabce39e9be48b7"}
Nov 02 02:47:30.765681 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:30.765653    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zs4kx" event={"ID":"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5","Type":"ContainerStarted","Data":"00262a1099ddb27b9e6fb19e76badf2e4fb526e77ed8e3e2395435f0086cf4f7"}
Nov 02 02:47:30.862982 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:30.862871    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:31.225732 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:31.225699    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:31.226109 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:31.225857    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:31.226109 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:31.225930    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:35.225912746 +0000 UTC m=+39.169409042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:31.770791 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:31.770748    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n4jz2" event={"ID":"149f4882-f029-488f-ba54-99d60c431352","Type":"ContainerStarted","Data":"589a3c33b986246f310ef4765136cb13e15bc591db5d0a555abb6b33922c2a92"}
Nov 02 02:47:31.883584 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:31.883524    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n4jz2" podStartSLOduration=1.6152142760000001 podStartE2EDuration="4.883506015s" totalImagesPullingTime="3.268291739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:31.883141627 +0000 UTC m=+35.826637936" watchObservedRunningTime="2026-11-02 02:47:31.883506015 +0000 UTC m=+35.827002326"
Nov 02 02:47:31.959216 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:31.959142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:32.104300 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:32.104221    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:34.780917 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:34.780877    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2h58d" event={"ID":"6090e8b3-5746-45d5-9a52-1b592b54afaf","Type":"ContainerStarted","Data":"8f443a1cf4244544b5f519f6b4e9e2c4798823b10bcc942c8c12afe3704a64bd"}
Nov 02 02:47:34.781466 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:34.781031    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:34.897195 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:34.897131    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2h58d" podStartSLOduration=33.691383835 podStartE2EDuration="37.897114176s" totalImagesPullingTime="4.205730341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:34.896468848 +0000 UTC m=+38.839965160" watchObservedRunningTime="2026-11-02 02:47:34.897114176 +0000 UTC m=+38.840610539"
Nov 02 02:47:35.259502 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.259456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:35.259691 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:35.259613    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:35.259741 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:35.259700    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:43.259683218 +0000 UTC m=+47.203179512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:35.266590 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.266564    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vzzk7"]
Nov 02 02:47:35.303768 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.303732    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:35.309462 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.309430    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.361965 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.361928    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vzzk7"]
Nov 02 02:47:35.368024 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.367999    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 02:47:35.368167 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.368090    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 02:47:35.368236 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.368194    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 02:47:35.410116 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.410087    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 02:47:35.410262 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.410145    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vvbct\""
Nov 02 02:47:35.461399 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.461369    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/850126c8-146a-4442-88d9-37e8f668e76e-data-volume\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.461556 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.461414    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/850126c8-146a-4442-88d9-37e8f668e76e-crio-socket\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.461556 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.461439    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.461556 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.461466    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.461681 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.461562    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fbxr\" (UniqueName: \"kubernetes.io/projected/850126c8-146a-4442-88d9-37e8f668e76e-kube-api-access-5fbxr\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.488004 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.487974    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"]
Nov 02 02:47:35.488643 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.488602    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:35.503147 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.503123    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:35.546523 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.546451    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"]
Nov 02 02:47:35.555939 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.555920    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 02:47:35.556048 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.555964    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 02:47:35.556807 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.556788    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-85n6h\""
Nov 02 02:47:35.562019 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.561996    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbxr\" (UniqueName: \"kubernetes.io/projected/850126c8-146a-4442-88d9-37e8f668e76e-kube-api-access-5fbxr\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.562117 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.562068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/850126c8-146a-4442-88d9-37e8f668e76e-data-volume\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.562181 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.562122    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/850126c8-146a-4442-88d9-37e8f668e76e-crio-socket\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.562181 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.562145    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.562276 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.562179    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.562276 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:35.562261    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 02:47:35.562369 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.562300    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/850126c8-146a-4442-88d9-37e8f668e76e-crio-socket\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.562369 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:35.562328    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:35.562369 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:35.562344    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:36.062325555 +0000 UTC m=+40.005821845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : configmap "kube-rbac-proxy" not found
Nov 02 02:47:35.562519 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:35.562402    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:36.062384481 +0000 UTC m=+40.005880769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:35.562519 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.562403    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/850126c8-146a-4442-88d9-37e8f668e76e-data-volume\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.624958 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.624925    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbxr\" (UniqueName: \"kubernetes.io/projected/850126c8-146a-4442-88d9-37e8f668e76e-kube-api-access-5fbxr\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:35.652881 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.652844    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:35.662974 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.662947    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwd84\" (UniqueName: \"kubernetes.io/projected/6fbda56c-15a7-4c75-bd74-92a331b780e1-kube-api-access-xwd84\") pod \"migrator-66df5d496b-q8tl6\" (UID: \"6fbda56c-15a7-4c75-bd74-92a331b780e1\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:35.763464 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.763429    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwd84\" (UniqueName: \"kubernetes.io/projected/6fbda56c-15a7-4c75-bd74-92a331b780e1-kube-api-access-xwd84\") pod \"migrator-66df5d496b-q8tl6\" (UID: \"6fbda56c-15a7-4c75-bd74-92a331b780e1\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:35.800001 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.799914    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:35.820095 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.820064    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwd84\" (UniqueName: \"kubernetes.io/projected/6fbda56c-15a7-4c75-bd74-92a331b780e1-kube-api-access-xwd84\") pod \"migrator-66df5d496b-q8tl6\" (UID: \"6fbda56c-15a7-4c75-bd74-92a331b780e1\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:35.829776 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.829752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:35.969557 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:35.969532    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:36.066587 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:36.066504    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:36.066587 ip-10-0-131-112 kubenswrapper[2574]: I1102 02:47:36.066541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:36.066779 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:36.066642    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 02:47:36.066779 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:36.066669    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:36.066779 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:36.066707    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:37.066689235 +0000 UTC m=+41.010185541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : configmap "kube-rbac-proxy" not found
Nov 02 02:47:36.066779 ip-10-0-131-112 kubenswrapper[2574]: E1102 02:47:36.066723    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:37.066717032 +0000 UTC m=+41.010213323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:36.132994 ip-10-0-131-112 systemd[1]: Stopping Kubernetes Kubelet...
Nov 02 02:47:36.147993 ip-10-0-131-112 systemd[1]: kubelet.service: Deactivated successfully.
Nov 02 02:47:36.148162 ip-10-0-131-112 systemd[1]: Stopped Kubernetes Kubelet.
Nov 02 02:47:36.148210 ip-10-0-131-112 systemd[1]: kubelet.service: Consumed 2.301s CPU time.
Nov 02 02:47:36.163161 ip-10-0-131-112 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 02:47:36.269009 ip-10-0-131-112 kubenswrapper[4407]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 02:47:36.269009 ip-10-0-131-112 kubenswrapper[4407]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 02:47:36.269009 ip-10-0-131-112 kubenswrapper[4407]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 02:47:36.269009 ip-10-0-131-112 kubenswrapper[4407]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 02:47:36.271183 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271103    4407 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:47:36.271183 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271183    4407 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271188    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271191    4407 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271194    4407 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271197    4407 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271200    4407 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271203    4407 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271206    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271208    4407 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271211    4407 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271213    4407 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271216    4407 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271219    4407 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271221    4407 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271223    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271227    4407 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271229    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271232    4407 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271234    4407 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:47:36.271254 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271237    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271240    4407 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271251    4407 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271254    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271257    4407 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271260    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271263    4407 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271266    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271269    4407 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271271    4407 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271274    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271276    4407 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271279    4407 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271281    4407 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271283    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271286    4407 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271288    4407 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271291    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271293    4407 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271296    4407 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:47:36.271728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271298    4407 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271301    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271303    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271306    4407 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271308    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271310    4407 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271313    4407 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271315    4407 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271318    4407 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271320    4407 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271325    4407 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271330    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271333    4407 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271335    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271339    4407 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271346    4407 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271349    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271352    4407 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271354    4407 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:47:36.272223 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271357    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271359    4407 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271362    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271364    4407 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271367    4407 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271369    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271372    4407 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271374    4407 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271377    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271379    4407 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271382    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271384    4407 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271387    4407 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271389    4407 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271392    4407 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271395    4407 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271397    4407 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271400    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271402    4407 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271405    4407 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:47:36.272707 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271407    4407 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271410    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271412    4407 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271415    4407 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271418    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271420    4407 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271423    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271426    4407 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271429    4407 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271431    4407 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271434    4407 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271437    4407 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271439    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271441    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271444    4407 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271447    4407 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271452    4407 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271456    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271459    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:47:36.273198 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271461    4407 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271464    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271466    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271469    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271471    4407 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271474    4407 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271476    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.271479    4407 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272237    4407 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272243    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272246    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272249    4407 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272251    4407 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272255    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272257    4407 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272260    4407 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272264    4407 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272266    4407 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272269    4407 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272272    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:47:36.273731 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272274    4407 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272277    4407 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272279    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272284    4407 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272288    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272291    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272294    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272297    4407 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272299    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272301    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272304    4407 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272307    4407 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272309    4407 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272311    4407 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272314    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272316    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272319    4407 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272321    4407 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272323    4407 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:47:36.274219 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272326    4407 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272328    4407 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272331    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272334    4407 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272337    4407 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272340    4407 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272342    4407 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272345    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272347    4407 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272355    4407 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272357    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272360    4407 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272362    4407 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272365    4407 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272367    4407 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272370    4407 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272372    4407 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272375    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272378    4407 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272380    4407 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:47:36.274703 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272383    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272385    4407 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272388    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272390    4407 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272393    4407 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272395    4407 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272397    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272400    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272402    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272406    4407 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272408    4407 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272410    4407 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272413    4407 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272415    4407 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272419    4407 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272421    4407 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272424    4407 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272426    4407 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272428    4407 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:47:36.275194 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272431    4407 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272434    4407 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272438    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272440    4407 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272443    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272445    4407 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272448    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272450    4407 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272453    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272455    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272458    4407 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272461    4407 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272463    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272466    4407 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272470    4407 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272473    4407 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272476    4407 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272479    4407 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272483    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:47:36.275674 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272486    4407 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272488    4407 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272491    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272494    4407 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272497    4407 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272499    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272502    4407 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272504    4407 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272507    4407 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272509    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272512    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272514    4407 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272517    4407 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272519    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272522    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272525    4407 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.272528    4407 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273013    4407 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273019    4407 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:47:36.276130 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273021    4407 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273024    4407 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273026    4407 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273031    4407 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273033    4407 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273036    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273039    4407 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273042    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273045    4407 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273047    4407 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273050    4407 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273052    4407 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273055    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273057    4407 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273060    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273062    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273065    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273067    4407 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273070    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273072    4407 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:47:36.276595 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273075    4407 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273077    4407 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273080    4407 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273083    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273085    4407 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273088    4407 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273090    4407 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273093    4407 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273097    4407 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273099    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273102    4407 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273104    4407 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273107    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273109    4407 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273111    4407 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273115    4407 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273118    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273121    4407 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273124    4407 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273126    4407 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:47:36.277248 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273128    4407 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273131    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273134    4407 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273136    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273139    4407 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273141    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273144    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273146    4407 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273149    4407 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273151    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273155    4407 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273158    4407 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273162    4407 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273164    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273167    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273170    4407 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273173    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273175    4407 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273177    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:47:36.278133 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273180    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273185    4407 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273187    4407 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273190    4407 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273194    4407 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273198    4407 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273201    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273205    4407 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273210    4407 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273214    4407 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273217    4407 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273222    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273226    4407 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273230    4407 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273233    4407 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273236    4407 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273240    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273244    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273248    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273252    4407 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:47:36.278998 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273256    4407 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273260    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273264    4407 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273267    4407 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273270    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273273    4407 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273275    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273280    4407 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273284    4407 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273287    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273289    4407 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273292    4407 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273295    4407 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273298    4407 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273301    4407 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273304    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273306    4407 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273310    4407 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273312    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:47:36.279602 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273315    4407 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273318    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273321    4407 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273324    4407 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273326    4407 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.273329    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273422    4407 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273430    4407 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273438    4407 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273444    4407 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273458    4407 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273462    4407 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273469    4407 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273473    4407 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273476    4407 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273479    4407 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273483    4407 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273486    4407 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273489    4407 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273492    4407 flags.go:64] FLAG: --cgroup-root=""
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273494    4407 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273497    4407 flags.go:64] FLAG: --client-ca-file=""
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273500    4407 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 02:47:36.280125 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273503    4407 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273507    4407 flags.go:64] FLAG: --cluster-domain=""
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273510    4407 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273515    4407 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273518    4407 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273521    4407 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273525    4407 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273528    4407 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273531    4407 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273534    4407 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273538    4407 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273541    4407 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273544    4407 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273547    4407 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273550    4407 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273554    4407 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273558    4407 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273561    4407 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273564    4407 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273567    4407 flags.go:64] FLAG: --enable-server="true"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273570    4407 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273574    4407 flags.go:64] FLAG: --event-burst="100"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273577    4407 flags.go:64] FLAG: --event-qps="50"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273580    4407 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 02:47:36.280690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273583    4407 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273585    4407 flags.go:64] FLAG: --eviction-hard=""
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273589    4407 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273592    4407 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273595    4407 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273598    4407 flags.go:64] FLAG: --eviction-soft=""
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273601    4407 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273604    4407 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273607    4407 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273610    4407 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273613    4407 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273617    4407 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273632    4407 flags.go:64] FLAG: --feature-gates=""
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273636    4407 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273639    4407 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273643    4407 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273646    4407 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273649    4407 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273656    4407 flags.go:64] FLAG: --help="false"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273659    4407 flags.go:64] FLAG: --hostname-override="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273662    4407 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273665    4407 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273668    4407 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 02:47:36.281292 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273671    4407 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273675    4407 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273678    4407 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273681    4407 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273684    4407 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273686    4407 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273689    4407 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273692    4407 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273695    4407 flags.go:64] FLAG: --kube-reserved=""
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273697    4407 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273700    4407 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273703    4407 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273706    4407 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273708    4407 flags.go:64] FLAG: --lock-file=""
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273711    4407 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273714    4407 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273717    4407 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273722    4407 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273725    4407 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273728    4407 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273732    4407 flags.go:64] FLAG: --logging-format="text"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273735    4407 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273739    4407 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273742    4407 flags.go:64] FLAG: --manifest-url=""
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273745    4407 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 02:47:36.281876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273749    4407 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273752    4407 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273757    4407 flags.go:64] FLAG: --max-pods="110"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273760    4407 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273763    4407 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273766    4407 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273768    4407 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273771    4407 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273774    4407 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273777    4407 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273785    4407 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273788    4407 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273791    4407 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273794    4407 flags.go:64] FLAG: --pod-cidr=""
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273797    4407 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273800    4407 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273803    4407 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273805    4407 flags.go:64] FLAG: --port="10250"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273808    4407 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273811    4407 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-016326884fc7c9386"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273814    4407 flags.go:64] FLAG: --qos-reserved=""
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273817    4407 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273819    4407 flags.go:64] FLAG: --register-node="true"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273822    4407 flags.go:64] FLAG: --register-with-taints=""
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273826    4407 flags.go:64] FLAG: --registry-burst="10"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273828    4407 flags.go:64] FLAG: --registry-qps="5"
Nov 02 02:47:36.282487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273831    4407 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273835    4407 flags.go:64] FLAG: --reserved-memory=""
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273838    4407 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273841    4407 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273845    4407 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273847    4407 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273850    4407 flags.go:64] FLAG: --runonce="false"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273853    4407 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273856    4407 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273859    4407 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273862    4407 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273865    4407 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273868    4407 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273871    4407 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273874    4407 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273877    4407 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273880    4407 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273883    4407 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273886    4407 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273889    4407 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273892    4407 flags.go:64] FLAG: --system-cgroups=""
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273894    4407 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273899    4407 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273902    4407 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273905    4407 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 02:47:36.283145 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273909    4407 flags.go:64] FLAG: --tls-min-version=""
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273912    4407 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273914    4407 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273917    4407 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273920    4407 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273923    4407 flags.go:64] FLAG: --v="2"
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273927    4407 flags.go:64] FLAG: --version="false"
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273931    4407 flags.go:64] FLAG: --vmodule=""
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273937    4407 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.273940    4407 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274071    4407 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274075    4407 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274080    4407 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274083    4407 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274085    4407 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274088    4407 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274091    4407 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274093    4407 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274096    4407 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274098    4407 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274101    4407 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:47:36.283876 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274103    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274106    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274108    4407 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274110    4407 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274113    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274116    4407 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274118    4407 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274121    4407 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274123    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274126    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274128    4407 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274131    4407 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274133    4407 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274136    4407 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274138    4407 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274141    4407 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274143    4407 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274146    4407 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274148    4407 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274153    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:47:36.284391 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274155    4407 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274158    4407 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274161    4407 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274163    4407 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274166    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274168    4407 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274171    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274173    4407 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274176    4407 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274179    4407 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274181    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274183    4407 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274186    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274189    4407 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274191    4407 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274193    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274196    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274199    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274201    4407 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:47:36.284958 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274204    4407 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274206    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274208    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274211    4407 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274213    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274216    4407 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274218    4407 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274221    4407 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274224    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274226    4407 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274229    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274231    4407 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274235    4407 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274239    4407 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274244    4407 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274247    4407 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274250    4407 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274252    4407 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274255    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:47:36.285472 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274258    4407 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274261    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274263    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274265    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274268    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274271    4407 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274273    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274276    4407 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274278    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274281    4407 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274283    4407 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274286    4407 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274292    4407 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274295    4407 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274298    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274301    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274304    4407 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274306    4407 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274309    4407 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274311    4407 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:47:36.286154 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274314    4407 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274316    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274319    4407 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274321    4407 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274324    4407 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274327    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274330    4407 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274334    4407 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274336    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274339    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274341    4407 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274344    4407 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274346    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274349    4407 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274351    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274354    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:47:36.286906 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274356    4407 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.274382    4407 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274420    4407 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274424    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274427    4407 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274430    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274432    4407 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274435    4407 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274437    4407 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274440    4407 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274443    4407 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274446    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274448    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274451    4407 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274454    4407 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274456    4407 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:47:36.287412 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274458    4407 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274461    4407 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274463    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274466    4407 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274468    4407 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274471    4407 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274474    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274478    4407 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274480    4407 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274483    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274486    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274488    4407 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274491    4407 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274493    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274496    4407 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274498    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274501    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274503    4407 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274506    4407 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:47:36.287839 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274510    4407 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274513    4407 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274516    4407 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274519    4407 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274522    4407 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274524    4407 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274526    4407 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274529    4407 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274532    4407 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274534    4407 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274537    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274539    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274542    4407 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274544    4407 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274546    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274549    4407 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274551    4407 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274554    4407 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274556    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274561    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:47:36.288471 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274565    4407 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274567    4407 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274570    4407 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274572    4407 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274575    4407 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274577    4407 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274580    4407 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274582    4407 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274584    4407 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274587    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274589    4407 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274592    4407 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274595    4407 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274598    4407 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274600    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274603    4407 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274605    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274608    4407 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274610    4407 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:47:36.289201 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274613    4407 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274616    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274631    4407 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274634    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274637    4407 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274639    4407 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274642    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274644    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274647    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274649    4407 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274652    4407 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274654    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274658    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274661    4407 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274664    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274666    4407 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274669    4407 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274671    4407 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274674    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:47:36.289810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274677    4407 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274679    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274682    4407 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274684    4407 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274689    4407 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274692    4407 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274696    4407 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274699    4407 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274701    4407 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274704    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274707    4407 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274710    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274712    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274715    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.274718    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.280529    4407 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.280543    4407 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.280563    4407 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.280567    4407 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 02:47:36.290291 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280733    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280740    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280750    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280753    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280756    4407 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280759    4407 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280762    4407 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280766    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280770    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280775    4407 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280778    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280783    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280788    4407 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280792    4407 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280809    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280814    4407 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280818    4407 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280823    4407 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280827    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280835    4407 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:47:36.290890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280978    4407 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280991    4407 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280994    4407 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.280997    4407 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281000    4407 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281003    4407 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281006    4407 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281009    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281012    4407 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281015    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281017    4407 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281021    4407 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281024    4407 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281027    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281029    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281032    4407 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281034    4407 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281037    4407 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281039    4407 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281042    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:47:36.291376 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281045    4407 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281048    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281051    4407 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281055    4407 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281057    4407 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281060    4407 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281063    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281065    4407 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281069    4407 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281072    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281074    4407 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281077    4407 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281080    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281082    4407 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281085    4407 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281088    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281090    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281093    4407 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281096    4407 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281098    4407 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:47:36.291890 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281101    4407 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281104    4407 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281106    4407 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281109    4407 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281112    4407 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281116    4407 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281120    4407 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281124    4407 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281127    4407 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281130    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281133    4407 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281135    4407 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281138    4407 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281140    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281143    4407 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281146    4407 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281148    4407 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281151    4407 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281153    4407 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:47:36.292385 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281165    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281169    4407 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281173    4407 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281177    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281180    4407 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281182    4407 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281185    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281187    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281190    4407 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281192    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281195    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281197    4407 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281200    4407 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281202    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281205    4407 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281207    4407 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281210    4407 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281212    4407 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281215    4407 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:47:36.292977 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281217    4407 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281219    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281222    4407 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281225    4407 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281227    4407 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281230    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281232    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281235    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.281274    4407 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281421    4407 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281426    4407 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281430    4407 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281434    4407 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281437    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281457    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:47:36.293459 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281461    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281464    4407 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281467    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281469    4407 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281471    4407 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281474    4407 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281476    4407 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281479    4407 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281481    4407 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281484    4407 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281486    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281488    4407 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281491    4407 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281493    4407 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281495    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281498    4407 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281501    4407 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281503    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281506    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281508    4407 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:47:36.293859 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281511    4407 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281513    4407 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281516    4407 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281519    4407 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281521    4407 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281524    4407 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281526    4407 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281529    4407 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281531    4407 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281533    4407 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281536    4407 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281539    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281547    4407 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281550    4407 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281552    4407 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281555    4407 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281557    4407 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281559    4407 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281562    4407 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:47:36.294337 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281564    4407 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281567    4407 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281569    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281572    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281574    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281576    4407 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281579    4407 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281581    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281584    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281586    4407 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281589    4407 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281592    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281595    4407 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281600    4407 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281603    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281605    4407 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281608    4407 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281610    4407 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281614    4407 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:47:36.294814 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281617    4407 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281641    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281645    4407 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281648    4407 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281651    4407 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281653    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281656    4407 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281664    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281667    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281670    4407 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281672    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281675    4407 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281677    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281680    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281682    4407 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281685    4407 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281687    4407 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281689    4407 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281692    4407 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281694    4407 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:47:36.295263 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281697    4407 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281699    4407 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281702    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281705    4407 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281707    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281710    4407 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281712    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281715    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281717    4407 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281720    4407 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281722    4407 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281725    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281727    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281730    4407 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281732    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281734    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281737    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281740    4407 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281742    4407 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:47:36.295756 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281745    4407 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281753    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281756    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.281782    4407 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281814    4407 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281818    4407 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281821    4407 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281825    4407 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281828    4407 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281830    4407 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281833    4407 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281836    4407 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281838    4407 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281841    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281844    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 02:47:36.296371 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281848    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281851    4407 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281856    4407 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281875    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281879    4407 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281882    4407 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281885    4407 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281888    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281892    4407 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281896    4407 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281900    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281902    4407 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281906    4407 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281909    4407 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281913    4407 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281916    4407 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281919    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281921    4407 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281924    4407 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 02:47:36.296979 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281935    4407 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281937    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281940    4407 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281942    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281945    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281948    4407 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281951    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281954    4407 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281956    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281959    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281961    4407 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281964    4407 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281967    4407 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281969    4407 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281972    4407 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281975    4407 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281978    4407 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281980    4407 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281983    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281985    4407 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 02:47:36.297838 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281988    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281990    4407 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281993    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281995    4407 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.281998    4407 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282000    4407 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282003    4407 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282005    4407 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282008    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282011    4407 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282014    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282016    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282018    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282027    4407 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282030    4407 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282033    4407 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282036    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282038    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282041    4407 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 02:47:36.298728 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282044    4407 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282046    4407 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282049    4407 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282052    4407 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282055    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282057    4407 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282060    4407 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282062    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282065    4407 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282067    4407 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282070    4407 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282072    4407 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282074    4407 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282077    4407 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282079    4407 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282082    4407 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282084    4407 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282086    4407 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282089    4407 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282092    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 02:47:36.299495 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282094    4407 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282097    4407 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282099    4407 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282102    4407 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282104    4407 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282108    4407 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282110    4407 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282118    4407 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282121    4407 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282124    4407 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282126    4407 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282129    4407 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282131    4407 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282134    4407 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282136    4407 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282139    4407 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:36.282142    4407 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.282222    4407 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.283204    4407 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 02 02:47:36.300348 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.283257    4407 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.283437    4407 server.go:1014] "Starting client certificate rotation"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.283576    4407 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.283640    4407 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 02:41:56 +0000 UTC" deadline="2028-07-10 10:15:18.456953204 +0000 UTC"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.283669    4407 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14791h27m42.173286094s"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.283936    4407 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.285055    4407 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.286666    4407 log.go:25] "Validated CRI v1 runtime API"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.288200    4407 log.go:25] "Validated CRI v1 image API"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.288936    4407 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 02:47:36.300972 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.290511    4407 fs.go:135] Filesystem UUIDs: map[0c96eaaa-0794-4dc5-8ffc-e9cd62359b22:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c50fc222-72a5-44f0-9c74-d7383b9840aa:/dev/nvme0n1p3]
Nov 02 02:47:36.301279 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.290529    4407 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/00262a1099ddb27b9e6fb19e76badf2e4fb526e77ed8e3e2395435f0086cf4f7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/00262a1099ddb27b9e6fb19e76badf2e4fb526e77ed8e3e2395435f0086cf4f7/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/00980703f376325d582c4f2157a928fdd9f8b8657cecbff59e46352d5e71daa5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/00980703f376325d582c4f2157a928fdd9f8b8657cecbff59e46352d5e71daa5/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/01f0668a0cc86cc92260ac27d61b4909d23f7d9988a28ead3c4a7206ed01f779/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/01f0668a0cc86cc92260ac27d61b4909d23f7d9988a28ead3c4a7206ed01f779/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/17150c480564a5c8773c9aa24ad3f72ca311899ce3fa8b24a23daa341fa2472d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/17150c480564a5c8773c9aa24ad3f72ca311899ce3fa8b24a23daa341fa2472d/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2aaef35ea454502d01836b9e66ed47e020027f675e56df6e84fe667043f80b84/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2aaef35ea454502d01836b9e66ed47e020027f675e56df6e84fe667043f80b84/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/594328116a7108957d4adb7600c006728ed8f54dae2fb0520293b1e26ca7b1db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/594328116a7108957d4adb7600c006728ed8f54dae2fb0520293b1e26ca7b1db/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5b381e938cb6a97e61e7854f32a8c58c4c368d844d8cfe0fde4f685b4e82c40d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5b381e938cb6a97e61e7854f32a8c58c4c368d844d8cfe0fde4f685b4e82c40d/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/802e7d2db4353e199fb67d5d3ac0334114278aa867ed8a0b4f9bd78a1543d8cf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/802e7d2db4353e199fb67d5d3ac0334114278aa867ed8a0b4f9bd78a1543d8cf/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81ae5c591b76617400b6e04b4fa6070fcd4ec7daf6815b4a4fe63f04b5f05e3b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81ae5c591b76617400b6e04b4fa6070fcd4ec7daf6815b4a4fe63f04b5f05e3b/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/83b2570bb7df51d94ef3cb804af0ded34fdcd2c845439b622bb3ab10c0f5f369/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/83b2570bb7df51d94ef3cb804af0ded34fdcd2c845439b622bb3ab10c0f5f369/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b7ee88a74b0b26072c9fcd0534e1ee22a726f6758c817131152357117a1bd974/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b7ee88a74b0b26072c9fcd0534e1ee22a726f6758c817131152357117a1bd974/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d2ee2156b68aacc4ae5745a6bd6b5f3a14c67984fa4649f701236156ae8df082/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d2ee2156b68aacc4ae5745a6bd6b5f3a14c67984fa4649f701236156ae8df082/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e638179b7cdf78468e34d16ac6f6ac761dc24a4c8dba543523f47936fe050e02/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e638179b7cdf78468e34d16ac6f6ac761dc24a4c8dba543523f47936fe050e02/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff186b20a78179e47a590d786c2cd5a59a325054f36d187dfcabce39e9be48b7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff186b20a78179e47a590d786c2cd5a59a325054f36d187dfcabce39e9be48b7/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/149f4882-f029-488f-ba54-99d60c431352/volumes/kubernetes.io~projected/kube-api-access-w9v8q:{mountpoint:/var/lib/kubelet/pods/149f4882-f029-488f-ba54-99d60c431352/volumes/kubernetes.io~projected/kube-api-access-w9v8q major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/149f4882-f029-488f-ba54-99d60c431352/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/149f4882-f029-488f-ba54-99d60c431352/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c995359-9608-4568-947c-b9464cf3c4c0/volumes/kubernetes.io~projected/kube-api-access-9znzz:{mountpoint:/var/lib/kubelet/pods/1c995359-9608-4568-947c-b9464cf3c4c0/volumes/kubernetes.io~projected/kube-api-access-9znzz major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2037171a-040e-4b0e-a635-a0b500821338/volumes/kubernetes.io~projected/kube-api-access-dtmk2:{mountpoint:/var/lib/kubelet/pods/2037171a-040e-4b0e-a635-a0b500821338/volumes/kubernetes.io~projected/kube-api-access-dtmk2 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/325964e5-52a3-4672-bda2-ed01ffeee604/volumes/kubernetes.io~projected/kube-api-access-hg7cc:{mountpoint:/var/lib/kubelet/pods/325964e5-52a3-4672-bda2-ed01ffeee604/volumes/kubernetes.io~projected/kube-api-access-hg7cc major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b9f52ac-c454-4aa5-bb64-3bdae890695d/volumes/kubernetes.io~projected/kube-api-access-6grpn:{mountpoint:/var/lib/kubelet/pods/3b9f52ac-c454-4aa5-bb64-3bdae890695d/volumes/kubernetes.io~projected/kube-api-access-6grpn major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6090e8b3-5746-45d5-9a52-1b592b54afaf/volumes/kubernetes.io~projected/kube-api-access-h5rmj:{mountpoint:/var/lib/kubelet/pods/6090e8b3-5746-45d5-9a52-1b592b54afaf/volumes/kubernetes.io~projected/kube-api-access-h5rmj major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fbda56c-15a7-4c75-bd74-92a331b780e1/volumes/kubernetes.io~projected/kube-api-access-xwd84:{mountpoint:/var/lib/kubelet/pods/6fbda56c-15a7-4c75-bd74-92a331b780e1/volumes/kubernetes.io~projected/kube-api-access-xwd84 major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77c73be0-a65c-490e-9c4e-e4c15006f5c8/volumes/kubernetes.io~projected/kube-api-access-mhjgx:{mountpoint:/var/lib/kubelet/pods/77c73be0-a65c-490e-9c4e-e4c15006f5c8/volumes/kubernetes.io~projected/kube-api-access-mhjgx major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7f3cbd14-2c73-4682-97d1-9beb1df2007e/volumes/kubernetes.io~projected/kube-api-access-lsjzw:{mountpoint:/var/lib/kubelet/pods/7f3cbd14-2c73-4682-97d1-9beb1df2007e/volumes/kubernetes.io~projected/kube-api-access-lsjzw major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ffe7d41-e27f-4373-8117-c49008015a88/volumes/kubernetes.io~projected/kube-api-access-25h4m:{mountpoint:/var/lib/kubelet/pods/7ffe7d41-e27f-4373-8117-c49008015a88/volumes/kubernetes.io~projected/kube-api-access-25h4m major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/850126c8-146a-4442-88d9-37e8f668e76e/volumes/kubernetes.io~projected/kube-api-access-5fbxr:{mountpoint:/var/lib/kubelet/pods/850126c8-146a-4442-88d9-37e8f668e76e/volumes/kubernetes.io~projected/kube-api-access-5fbxr major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~projected/kube-api-access-vwzpc:{mountpoint:/var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~projected/kube-api-access-vwzpc major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ee88ccb5-4a04-431a-8963-7b8c7393604a/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ee88ccb5-4a04-431a-8963-7b8c7393604a/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volumes/kubernetes.io~projected/kube-api-access-9nhqj:{mountpoint:/var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volumes/kubernetes.io~projected/kube-api-access-9nhqj major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb902c0e-8074-4363-90c5-04eaa2c82fb3/volumes/kubernetes.io~projected/kube-api-access-97v74:{mountpoint:/var/lib/kubelet/pods/fb902c0e-8074-4363-90c5-04eaa2c82fb3/volumes/kubernetes.io~projected/kube-api-access-97v74 major:0 minor:55 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/67982aba4a7c5ea0f2e3d7c0963327b2525480b7c514bba99988383857d49250/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/310e9713e0e49aea491d9163dac17b114b627e0e0833dcc104c357e344877b8f/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/014033d832f4e2e77fba27b77076a9f6054dcd8365db6624bbf50335b60dda1f/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-107:{mountpoint:/var/lib/containers/storage/overlay/3291e67882eabccb26038000fb1694e676c8c7c8379837b256c707b11482ab73/merged major:0 minor:107 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/f50e3c5fbfc9a17b96b1ed5de32747bfc576c14b436b1600f3431c50bb1a253c/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-116:{mountpoint:/var/lib/containers/storage/overlay/e9aba8f13e446628970c3a0e37b867285b170abff41c792ff97ceb6b72568c64/merged major:0 minor:116 fsType:overlay blockSize:0} overlay_0-121:{mountpoint:/var/lib/containers/storage/overlay/fe55de1276017737229d103c808f2a855739b2563c72904b1b611061fc6ec3a1/merged major:0 minor:121 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/d3940b04a8d0c4df3b8303c28bea1b8a2bfbe993ed1d573f38eed5ca4a83852e/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/b5999591e70a56eacbc12168964faccdb48daa1f755e126159bd94505d3e1da9/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/6cc7cd55ef024c796f17537bdf14131c85efd009d52e1e693465f9c51f98b285/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/fa365f9a1b790a7313409d2cea75a5a59cdba79328358f38a26e29496c9b044f/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/c21f05c13ae792546d965aa6cebf1ee1dd8388ed9e9be0c8c7b057a15f735e7a/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/155abe1bbcd72a72418d584c057f08ef05d52f3ef07fcab902baab46d617fec8/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/b444651aa3f69622e7337748dbf3cf0e57c0322f89c58f2b7e992e23b6335e6a/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/4dfebdf4b4c40cc36f63156932f171960941f95fa5a8d3ca36cf810450cf3ce0/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/21ff3cbec7d203f79c7882c5507f9e76b304f9f743be114e79cbc1c4108f6f1c/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/757d63a14b6c05c2043071a541f6fb9982e7b37fc62a8cea94859318a3f1aabb/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/c5ab11d10ca3acdf67eb89bf0adf8f71d4756185ef85913dcc3f17cf18b7b02e/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/1c33ae3215bfa3be34df424f2896a2e3f4cec38760072935832a780805d770f8/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/2d847d72e267b0b9284a088bf70eb4a4d0ae971259c896dbb1ba7e5a35688c1c/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-189:{mountpoint:/var/lib/containers/storage/overlay/d2673ff5d7791cf9252a68e6b9bef2880c84c9e71a8348a5216b5c2fb4b96777/merged major:0 minor:189 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/632b3f36394448cc9258372e05a11a31b45a6f510285a5cbb19f25cecd0601da/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/f62e00d4fdd0d4eabc281bb56f3036c437dc6a56d94f0d8c26f9f34f7ecf6c65/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-214:{mountpoint:/var/lib/containers/storage/overlay/22f235adf98a1bc3dd29ca19adecff1bfc96fa59397a47a22d161c8f39d76eee/merged major:0 minor:214 fsType:overlay blockSize:0} overlay_0-216:{mountpoint:/var/lib/containers/storage/overlay/e47e5acc798c093b65058db0340e68d8be19bdbc6983ce28b90c531d18be6865/merged major:0 minor:216 fsType:overlay blockSize:0} overlay_0-218:{mountpoint:/var/lib/containers/storage/overlay/fa48708853f5f839123a5e1516c2912518ac40160229580640a4c84c66175196/merged major:0 minor:218 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/3396407eec066f6f94a9ce81687957d7d982c2206a16ff93275340c4928fb30d/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-46:{mountpoint:/var/lib/containers/storage/overlay/d59d7ecfd040acc89258a726efb25673f08d2ef30d29997ecc8e196e58af0641/merged major:0 minor:46 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/0f7f4cba34c4791dff86bdc4bfe32fd4aa91d07745ce6fe08bb24c12853ef102/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/4f2da8975c2d46ddd06d6a37548f08afb6181df942e00c1705ec1a43182206ea/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/4b80cfa1377d89879d5c6a96034694d436c2a37983229f0f3a5b5813bd69ef95/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/ed579a9b9e89b91ebb6a7c50990627415a38f60ae0465b46aac7f3510dfc00e1/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/9a46ce321a994075be41017e68ecab4966f9b2345b8a999132b7a73d9ea9c405/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/98290e0c18bd536417d19132421ec9eb1ad2a53680c1a372272177d8a05bc5d9/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/64a8498036a8bbdbf10262efd6bdbac14b2163b7352f2c79f4a6d27bb4bad278/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/9db8b9333d5022ca5b846533764b7a1b6c938c9b7651a795ddbfb1f75351745e/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/fff07353d29aeb956d4a0c0230b081f8f7623ad392b893125148a08016bba169/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/4ab1c51a28751c61e52878d428d88f86cd805262736aa9bb87ec3717400a6cb4/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.297601    4407 manager.go:217] Machine: {Timestamp:2026-11-02 02:47:36.296585526 +0000 UTC m=+0.091934084 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3076553 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208a7391c333b55d569b729662989f SystemUUID:ec208a73-91c3-33b5-5d56-9b729662989f BootID:5c03ba59-a253-4d0b-aaa1-190bf2694a69 Filesystems:[{Device:/run/containers/storage/overlay-containers/17150c480564a5c8773c9aa24ad3f72ca311899ce3fa8b24a23daa341fa2472d/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d2ee2156b68aacc4ae5745a6bd6b5f3a14c67984fa4649f701236156ae8df082/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~projected/kube-api-access-vwzpc DeviceMajor:0 DeviceMinor:53 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1c995359-9608-4568-947c-b9464cf3c4c0/volumes/kubernetes.io~projected/kube-api-access-9znzz DeviceMajor:0 DeviceMinor:54 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6fbda56c-15a7-4c75-bd74-92a331b780e1/volumes/kubernetes.io~projected/kube-api-access-xwd84 DeviceMajor:0 DeviceMinor:229 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:48 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/00980703f376325d582c4f2157a928fdd9f8b8657cecbff59e46352d5e71daa5/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-216 DeviceMajor:0 DeviceMinor:216 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/83b2570bb7df51d94ef3cb804af0ded34fdcd2c845439b622bb3ab10c0f5f369/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/802e7d2db4353e199fb67d5d3ac0334114278aa867ed8a0b4f9bd78a1543d8cf/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e638179b7cdf78468e34d16ac6f6ac761dc24a4c8dba543523f47936fe050e02/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-116 DeviceMajor:0 DeviceMinor:116 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-189 DeviceMajor:0 DeviceMinor:189 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/149f4882-f029-488f-ba54-99d60c431352/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-218 DeviceMajor:0 DeviceMinor:218 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2037171a-040e-4b0e-a635-a0b500821338/volumes/kubernetes.io~projected/kube-api-access-dtmk2 DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-107 DeviceMajor:0 DeviceMinor:107 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb902c0e-8074-4363-90c5-04eaa2c82fb3/volumes/kubernetes.io~projected/kube-api-access-97v74 DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volumes/kubernetes.io~projected/kube-api-access-9nhqj DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6090e8b3-5746-45d5-9a52-1b592b54afaf/volumes/kubernetes.io~projected/kube-api-access-h5rmj DeviceMajor:0 DeviceMinor:207 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-46 DeviceMajor:0 DeviceMinor:46 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/00262a1099ddb27b9e6fb19e76badf2e4fb526e77ed8e3e2395435f0086cf4f7/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/850126c8-146a-4442-88d9-37e8f668e76e/volumes/kubernetes.io~projected/kube-api-access-5fbxr DeviceMajor:0 DeviceMinor:224 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/01f0668a0cc86cc92260ac27d61b4909d23f7d9988a28ead3c4a7206ed01f779/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7f3cbd14-2c73-4682-97d1-9beb1df2007e/volumes/kubernetes.io~projected/kube-api-access-lsjzw DeviceMajor:0 DeviceMinor:51 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/594328116a7108957d4adb7600c006728ed8f54dae2fb0520293b1e26ca7b1db/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff186b20a78179e47a590d786c2cd5a59a325054f36d187dfcabce39e9be48b7/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ee88ccb5-4a04-431a-8963-7b8c7393604a/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81ae5c591b76617400b6e04b4fa6070fcd4ec7daf6815b4a4fe63f04b5f05e3b/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f9913c84-a163-4a5b-b169-cc80806bc03a/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-214 DeviceMajor:0 DeviceMinor:214 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/325964e5-52a3-4672-bda2-ed01ffeee604/volumes/kubernetes.io~projected/kube-api-access-hg7cc DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/149f4882-f029-488f-ba54-99d60c431352/volumes/kubernetes.io~projected/kube-api-access-w9v8q DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:206 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7ffe7d41-e27f-4373-8117-c49008015a88/volumes/kubernetes.io~projected/kube-api-access-25h4m DeviceMajor:0 DeviceMinor:52 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2aaef35ea454502d01836b9e66ed47e020027f675e56df6e84fe667043f80b84/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3b9f52ac-c454-4aa5-bb64-3bdae890695d/volumes/kubernetes.io~projected/kube-api-access-6grpn DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5b381e938cb6a97e61e7854f32a8c58c4c368d844d8cfe0fde4f685b4e82c40d/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6ec759-f178-45cc-9c8e-7f5982c233dc/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-121 DeviceMajor:0 DeviceMinor:121 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77c73be0-a65c-490e-9c4e-e4c15006f5c8/volumes/kubernetes.io~projected/kube-api-access-mhjgx DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b7ee88a74b0b26072c9fcd0534e1ee22a726f6758c817131152357117a1bd974/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:00262a1099ddb27 MacAddress:56:1f:42:03:de:f7 Speed:10000 Mtu:8901} {Name:00980703f376325 MacAddress:8e:94:33:e0:0a:e4 Speed:10000 Mtu:8901} {Name:5b381e938cb6a97 MacAddress:8e:76:c9:ae:cc:d6 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:3d:e5:80:1a:35 Speed:0 Mtu:9001} {Name:br-int MacAddress:3e:65:1f:54:41:b5 Speed:0 Mtu:8901} {Name:ens5 MacAddress:02:3d:e5:80:1a:35 Speed:0 Mtu:9001} {Name:ff186b20a78179e MacAddress:76:9a:36:9a:14:b4 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:d2:60:e3:84:43:3f Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:26:32:e9:59:3b:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298059    4407 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298117    4407 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298500    4407 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298522    4407 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-112.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298725    4407 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298738    4407 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298758    4407 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298784    4407 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.298899    4407 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.299105    4407 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.299236    4407 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.299284    4407 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.299296    4407 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.299309    4407 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.299324    4407 kubelet.go:405] "Adding apiserver pod source"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.299335    4407 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.300081    4407 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.300093    4407 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 02:47:36.301409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.300641    4407 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 02:47:36.302519 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302505    4407 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 02:47:36.302519 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302518    4407 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 02:47:36.302617 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302607    4407 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 02 02:47:36.302818 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302807    4407 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 02:47:36.302852 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302836    4407 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 02:47:36.302926 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302917    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 02:47:36.302953 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302931    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 02:47:36.302953 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302937    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 02:47:36.302953 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302942    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 02:47:36.302953 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302948    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 02:47:36.302953 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302954    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 02:47:36.303094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302960    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 02:47:36.303094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302966    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 02:47:36.303094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302972    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 02:47:36.303094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302978    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 02:47:36.303094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.302991    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 02:47:36.303094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.303000    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 02:47:36.303094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.303025    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 02:47:36.303094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.303030    4407 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 02:47:36.306231 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.306218    4407 server.go:1263] "Started kubelet"
Nov 02 02:47:36.309035 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.306330    4407 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 02:47:36.309035 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.306401    4407 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 02:47:36.309035 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.306725    4407 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 02:47:36.309035 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.307032    4407 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 02:47:36.309035 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.308239    4407 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 02:47:36.306821 ip-10-0-131-112 systemd[1]: Started Kubernetes Kubelet.
Nov 02 02:47:36.309690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.309673    4407 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 02:47:36.309798 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.309744    4407 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 02:47:36.309982 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.309963    4407 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 02:47:36.310063 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.309984    4407 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 02:47:36.310063 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.310011    4407 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 02:47:36.310158 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.309950    4407 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 02:41:57 +0000 UTC" deadline="2028-04-21 03:09:04.246977857 +0000 UTC"
Nov 02 02:47:36.310158 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.310083    4407 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12864h21m27.936899628s"
Nov 02 02:47:36.310698 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.310611    4407 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-112.us-west-1.compute.internal\" not found"
Nov 02 02:47:36.313660 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313591    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b9f52ac-c454-4aa5-bb64-3bdae890695d" volumeName="kubernetes.io/empty-dir/3b9f52ac-c454-4aa5-bb64-3bdae890695d-tmp-dir" seLinuxMountContext=""
Nov 02 02:47:36.313660 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313652    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6ec759-f178-45cc-9c8e-7f5982c233dc" volumeName="kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-tmp" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313666    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9913c84-a163-4a5b-b169-cc80806bc03a" volumeName="kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-script-lib" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313687    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb902c0e-8074-4363-90c5-04eaa2c82fb3" volumeName="kubernetes.io/configmap/fb902c0e-8074-4363-90c5-04eaa2c82fb3-serviceca" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313700    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ffe7d41-e27f-4373-8117-c49008015a88" volumeName="kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-multus-daemon-config" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313712    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ffe7d41-e27f-4373-8117-c49008015a88" volumeName="kubernetes.io/projected/7ffe7d41-e27f-4373-8117-c49008015a88-kube-api-access-25h4m" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313729    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ee88ccb5-4a04-431a-8963-7b8c7393604a" volumeName="kubernetes.io/secret/ee88ccb5-4a04-431a-8963-7b8c7393604a-agent-certs" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313742    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77c73be0-a65c-490e-9c4e-e4c15006f5c8" volumeName="kubernetes.io/projected/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kube-api-access-mhjgx" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313763    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c995359-9608-4568-947c-b9464cf3c4c0" volumeName="kubernetes.io/projected/1c995359-9608-4568-947c-b9464cf3c4c0-kube-api-access-9znzz" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313777    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="325964e5-52a3-4672-bda2-ed01ffeee604" volumeName="kubernetes.io/empty-dir/325964e5-52a3-4672-bda2-ed01ffeee604-tmp-dir" seLinuxMountContext=""
Nov 02 02:47:36.313806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313789    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="325964e5-52a3-4672-bda2-ed01ffeee604" volumeName="kubernetes.io/projected/325964e5-52a3-4672-bda2-ed01ffeee604-kube-api-access-hg7cc" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313835    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="367d8f5b-1f4f-4d94-a9aa-7b007cd260a5" volumeName="kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313850    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f3cbd14-2c73-4682-97d1-9beb1df2007e" volumeName="kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313867    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb902c0e-8074-4363-90c5-04eaa2c82fb3" volumeName="kubernetes.io/projected/fb902c0e-8074-4363-90c5-04eaa2c82fb3-kube-api-access-97v74" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313887    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b9f52ac-c454-4aa5-bb64-3bdae890695d" volumeName="kubernetes.io/projected/3b9f52ac-c454-4aa5-bb64-3bdae890695d-kube-api-access-6grpn" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313905    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6090e8b3-5746-45d5-9a52-1b592b54afaf" volumeName="kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313917    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="149f4882-f029-488f-ba54-99d60c431352" volumeName="kubernetes.io/projected/149f4882-f029-488f-ba54-99d60c431352-kube-api-access-w9v8q" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313933    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="149f4882-f029-488f-ba54-99d60c431352" volumeName="kubernetes.io/secret/149f4882-f029-488f-ba54-99d60c431352-cert" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313945    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b9f52ac-c454-4aa5-bb64-3bdae890695d" volumeName="kubernetes.io/configmap/3b9f52ac-c454-4aa5-bb64-3bdae890695d-config-volume" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313956    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f3cbd14-2c73-4682-97d1-9beb1df2007e" volumeName="kubernetes.io/projected/7f3cbd14-2c73-4682-97d1-9beb1df2007e-kube-api-access-lsjzw" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.313987    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="850126c8-146a-4442-88d9-37e8f668e76e" volumeName="kubernetes.io/projected/850126c8-146a-4442-88d9-37e8f668e76e-kube-api-access-5fbxr" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314000    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6ec759-f178-45cc-9c8e-7f5982c233dc" volumeName="kubernetes.io/projected/8b6ec759-f178-45cc-9c8e-7f5982c233dc-kube-api-access-vwzpc" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314016    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2037171a-040e-4b0e-a635-a0b500821338" volumeName="kubernetes.io/projected/2037171a-040e-4b0e-a635-a0b500821338-kube-api-access-dtmk2" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314027    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ee88ccb5-4a04-431a-8963-7b8c7393604a" volumeName="kubernetes.io/configmap/ee88ccb5-4a04-431a-8963-7b8c7393604a-konnectivity-ca" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314038    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9913c84-a163-4a5b-b169-cc80806bc03a" volumeName="kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-config" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314054    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9913c84-a163-4a5b-b169-cc80806bc03a" volumeName="kubernetes.io/projected/f9913c84-a163-4a5b-b169-cc80806bc03a-kube-api-access-9nhqj" seLinuxMountContext=""
Nov 02 02:47:36.314070 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314065    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6ec759-f178-45cc-9c8e-7f5982c233dc" volumeName="kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-tuned" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314082    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f3cbd14-2c73-4682-97d1-9beb1df2007e" volumeName="kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-binary-copy" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314100    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ffe7d41-e27f-4373-8117-c49008015a88" volumeName="kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-cni-binary-copy" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314117    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9913c84-a163-4a5b-b169-cc80806bc03a" volumeName="kubernetes.io/secret/f9913c84-a163-4a5b-b169-cc80806bc03a-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314129    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2037171a-040e-4b0e-a635-a0b500821338" volumeName="kubernetes.io/configmap/2037171a-040e-4b0e-a635-a0b500821338-iptables-alerter-script" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314141    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fbda56c-15a7-4c75-bd74-92a331b780e1" volumeName="kubernetes.io/projected/6fbda56c-15a7-4c75-bd74-92a331b780e1-kube-api-access-xwd84" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314148    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f3cbd14-2c73-4682-97d1-9beb1df2007e" volumeName="kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314158    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="850126c8-146a-4442-88d9-37e8f668e76e" volumeName="kubernetes.io/empty-dir/850126c8-146a-4442-88d9-37e8f668e76e-data-volume" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314169    4407 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9913c84-a163-4a5b-b169-cc80806bc03a" volumeName="kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-env-overrides" seLinuxMountContext=""
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314177    4407 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 02:47:36.314471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314182    4407 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 02:47:36.314841 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314821    4407 factory.go:55] Registering systemd factory
Nov 02 02:47:36.314896 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.314846    4407 factory.go:223] Registration of the systemd container factory successfully
Nov 02 02:47:36.315117 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.315092    4407 factory.go:153] Registering CRI-O factory
Nov 02 02:47:36.315155 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.315122    4407 factory.go:223] Registration of the crio container factory successfully
Nov 02 02:47:36.315188 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.315175    4407 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 02:47:36.315220 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.315192    4407 factory.go:103] Registering Raw factory
Nov 02 02:47:36.315220 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.315208    4407 manager.go:1199] Started watching for new ooms in manager
Nov 02 02:47:36.315544 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.315530    4407 manager.go:319] Starting recovery of all containers
Nov 02 02:47:36.324815 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.324613    4407 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 02:47:36.343701 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.343672    4407 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 02:47:36.343701 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.343702    4407 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 02:47:36.343892 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.343724    4407 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 02:47:36.343892 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.343764    4407 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 02 02:47:36.353406 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.353360    4407 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="09f67c1e5d2274e6d58ce21752f8afd23b1648deb46d893ee0760001bfa6b6e8" exitCode=0
Nov 02 02:47:36.353406 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.353404    4407 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="78b74a5e46d6e4f48029877a0e935094a2545b80461f3e1da94e628b6e990ebd" exitCode=0
Nov 02 02:47:36.353563 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.353413    4407 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="c69ef9f55f8d48b9301431c292bdc657f836b2df47af2b032c714c1ae4defa8a" exitCode=0
Nov 02 02:47:36.353563 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.353420    4407 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="2250986d8fc2b6696dac9a5629bf9de3e495bab1b950440495cd0984fde6ab71" exitCode=0
Nov 02 02:47:36.353563 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.353429    4407 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="41438c9ad61ea3e11c231d542aa0d17022721a0010da6c6380db268e48ada882" exitCode=0
Nov 02 02:47:36.353563 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.353436    4407 generic.go:356] "Generic (PLEG): container finished" podID="7f3cbd14-2c73-4682-97d1-9beb1df2007e" containerID="b39d9b2fddde2f5ad3c2090481d85895fa21009b88b968042013a4101b427e22" exitCode=0
Nov 02 02:47:36.370096 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.370061    4407 generic.go:356] "Generic (PLEG): container finished" podID="832ceb98d0ffc61a89e76de2f575c0c6" containerID="d36bbbe18bf3b260026bdf406306a60ca3cfdf4737333be6f4102c7396f2789b" exitCode=0
Nov 02 02:47:36.378049 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.378032    4407 manager.go:324] Recovery completed
Nov 02 02:47:36.391773 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.391753    4407 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:47:36.392808 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.392787    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:47:36.392900 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.392822    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:47:36.392900 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.392837    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:47:36.394058 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394039    4407 cpu_manager.go:226] "Starting" policy="none"
Nov 02 02:47:36.394058 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394056    4407 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 02:47:36.394058 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394057    4407 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:47:36.394233 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394074    4407 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 02:47:36.394281 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394231    4407 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 02 02:47:36.394281 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394243    4407 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 02 02:47:36.394281 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394261    4407 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 02 02:47:36.394281 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394266    4407 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 02 02:47:36.394281 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394271    4407 policy_none.go:50] "Start"
Nov 02 02:47:36.394281 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394279    4407 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 02:47:36.394463 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394289    4407 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 02:47:36.394463 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394410    4407 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 02 02:47:36.394463 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394417    4407 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 02 02:47:36.394463 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394423    4407 policy_none.go:44] "Start"
Nov 02 02:47:36.394887 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394873    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:47:36.394941 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394900    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:47:36.394941 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.394910    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:47:36.398447 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.398423    4407 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 02:47:36.400864 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.400853    4407 manager.go:341] "Starting Device Plugin manager"
Nov 02 02:47:36.400921 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.400880    4407 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 02:47:36.400921 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.400890    4407 server.go:87] "Starting device plugin registration server"
Nov 02 02:47:36.401182 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.401169    4407 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 02:47:36.401216 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.401187    4407 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 02:47:36.401269 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.401256    4407 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 02:47:36.401374 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.401340    4407 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 02:47:36.401444 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.401379    4407 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 02:47:36.401444 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.401383    4407 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 02:47:36.401552 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.401420    4407 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T02:47:36.401370496Z","UUID":"ac20ff54-672b-4e3b-a306-f1a9026bb6ad","Handler":null,"Name":"","Endpoint":""}
Nov 02 02:47:36.402693 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.402671    4407 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 02:47:36.402770 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.402715    4407 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-112.us-west-1.compute.internal\" not found"
Nov 02 02:47:36.402770 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.402731    4407 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 02:47:36.402770 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.402751    4407 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 02:47:36.444308 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.443859    4407 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"]
Nov 02 02:47:36.444405 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.444383    4407 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:47:36.445400 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.445385    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:47:36.445494 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.445414    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:47:36.445494 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.445423    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:47:36.445677 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.445666    4407 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:47:36.445836 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.445822    4407 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:47:36.446419 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446402    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:47:36.446419 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446412    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:47:36.446515 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446436    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:47:36.446515 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446448    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:47:36.446515 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446437    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:47:36.446613 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446517    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:47:36.446613 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.446588    4407 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-112.us-west-1.compute.internal\" not found" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.446830 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446796    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"832ceb98d0ffc61a89e76de2f575c0c6","Type":"ContainerStarted","Data":"5f5dbcfc65a07fbe0177081ae6c217fd2d6d0a5d1cf99322ef2e4ca1a8392a14"}
Nov 02 02:47:36.446866 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446843    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"832ceb98d0ffc61a89e76de2f575c0c6","Type":"ContainerDied","Data":"d36bbbe18bf3b260026bdf406306a60ca3cfdf4737333be6f4102c7396f2789b"}
Nov 02 02:47:36.446894 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446863    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"832ceb98d0ffc61a89e76de2f575c0c6","Type":"ContainerStarted","Data":"83b2570bb7df51d94ef3cb804af0ded34fdcd2c845439b622bb3ab10c0f5f369"}
Nov 02 02:47:36.446894 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446880    4407 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:47:36.446960 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446899    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"9fdb79d33d08466df57752003cd13435","Type":"ContainerStarted","Data":"8bd8367fad8df8e5525b96f5d5ff6dcfebf53ee84c344ff759dd1ab4491fd1fc"}
Nov 02 02:47:36.446960 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.446918    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal" event={"ID":"9fdb79d33d08466df57752003cd13435","Type":"ContainerStarted","Data":"b7ee88a74b0b26072c9fcd0534e1ee22a726f6758c817131152357117a1bd974"}
Nov 02 02:47:36.447787 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.447774    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:47:36.447852 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.447799    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:47:36.447852 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.447810    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:47:36.447935 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.447927    4407 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-112.us-west-1.compute.internal\" not found" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.501687 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.501655    4407 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 02:47:36.502673 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.502646    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 02:47:36.502770 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.502685    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 02:47:36.502770 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.502695    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 02:47:36.502770 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.502715    4407 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.511217 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.511190    4407 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:47:36.511346 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.511230    4407 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:47:36.513858 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.513838    4407 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 02 02:47:36.559589 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.559540    4407 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:47:36.564474 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.564454    4407 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 02:47:36.611296 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.611224    4407 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.614412 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.614387    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.614490 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.614439    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.614490 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.614458    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fdb79d33d08466df57752003cd13435-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"9fdb79d33d08466df57752003cd13435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.656808 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.656779    4407 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.656915 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.656844    4407 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.657423 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.657408    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeNotReady"
Nov 02 02:47:36.657505 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.657438    4407 setters.go:546] "Node became not ready" node="ip-10-0-131-112.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-02T02:47:36Z","lastTransitionTime":"2026-11-02T02:47:36Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 02 02:47:36.662373 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.662351    4407 nodeinfomanager.go:471] existing CSINode "ip-10-0-131-112.us-west-1.compute.internal" is owned by different node (oldNodeID="bd12bc44-03b7-4ce3-95bc-4c1e851056a9", newNodeID=""), cleaning up...
Nov 02 02:47:36.670768 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.670749    4407 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 02:47:36.670831 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.670791    4407 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.670831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.670805    4407 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.715053 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.715025    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fdb79d33d08466df57752003cd13435-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"9fdb79d33d08466df57752003cd13435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.715160 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.715057    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.715160 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.715075    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.715160 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.715119    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.715160 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.715146    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fdb79d33d08466df57752003cd13435-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"9fdb79d33d08466df57752003cd13435\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.715332 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.715150    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/832ceb98d0ffc61a89e76de2f575c0c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" (UID: \"832ceb98d0ffc61a89e76de2f575c0c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:36.773315 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:36.773289    4407 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 02:47:36.773432 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:36.773331    4407 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:37.300223 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.300194    4407 apiserver.go:52] "Watching apiserver"
Nov 02 02:47:37.377600 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.377574    4407 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:37.377750 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.377692    4407 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:37.405231 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.405212    4407 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 02:47:37.405667 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.405617    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n4jz2","openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6","kube-system/global-pull-secret-syncer-zs4kx","kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal","openshift-network-diagnostics/network-check-target-2h58d","openshift-cluster-node-tuning-operator/tuned-q8k8l","kube-system/konnectivity-agent-b5wvk","openshift-insights/insights-runtime-extractor-vzzk7","openshift-multus/multus-bvqdp","openshift-ovn-kubernetes/ovnkube-node-4bpfv","openshift-multus/multus-additional-cni-plugins-2rm9b","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd","openshift-dns/node-resolver-z7p4b","openshift-image-registry/node-ca-mdddz","openshift-multus/network-metrics-daemon-4xrlc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-fbwhk","openshift-dns/dns-default-st5ft"]
Nov 02 02:47:37.407938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.407917    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:37.412442 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.412422    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.412802 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.412784    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:37.413184 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.413164    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.419563 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419543    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97v74\" (UniqueName: \"kubernetes.io/projected/fb902c0e-8074-4363-90c5-04eaa2c82fb3-kube-api-access-97v74\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:47:37.419655 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419570    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjzw\" (UniqueName: \"kubernetes.io/projected/7f3cbd14-2c73-4682-97d1-9beb1df2007e-kube-api-access-lsjzw\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.419655 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419590    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-log-socket\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.419655 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419615    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-script-lib\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.419768 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419677    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.419768 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419702    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.419768 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419720    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.419768 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419737    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9913c84-a163-4a5b-b169-cc80806bc03a-ovn-node-metrics-cert\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.419768 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419753    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-k8s-cni-cncf-io\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.419947 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419858    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-etc-kubernetes\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.419947 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419883    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.419947 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419898    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-registration-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.420029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419951    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.420029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.419986    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-var-lib-kubelet\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.420085 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420029    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-tuned\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.420085 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420077    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-var-lib-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.420137 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420105    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.420137 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420107    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-tuned\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.420197 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420146    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhqj\" (UniqueName: \"kubernetes.io/projected/f9913c84-a163-4a5b-b169-cc80806bc03a-kube-api-access-9nhqj\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.420197 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420164    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb902c0e-8074-4363-90c5-04eaa2c82fb3-host\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:47:37.420197 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420182    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:37.420287 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420204    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysconfig\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.420287 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420280    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-systemd\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.420358 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420297    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwzpc\" (UniqueName: \"kubernetes.io/projected/8b6ec759-f178-45cc-9c8e-7f5982c233dc-kube-api-access-vwzpc\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.420358 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420312    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9znzz\" (UniqueName: \"kubernetes.io/projected/1c995359-9608-4568-947c-b9464cf3c4c0-kube-api-access-9znzz\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:37.420358 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420327    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.420358 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420343    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-dbus\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:37.420532 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420358    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fb902c0e-8074-4363-90c5-04eaa2c82fb3-serviceca\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:47:37.420532 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420481    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-cni-binary-copy\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.420532 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420521    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-bin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.420721 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420543    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-hostroot\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.420721 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420558    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-conf-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.420721 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420581    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.420721 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420642    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-system-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.420721 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420663    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-multus-certs\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.420721 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420716    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25h4m\" (UniqueName: \"kubernetes.io/projected/7ffe7d41-e27f-4373-8117-c49008015a88-kube-api-access-25h4m\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.420938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420746    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-kubernetes\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.420938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420768    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-systemd-units\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.420938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420816    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-slash\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.420938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420865    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-ovn\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.420938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420890    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2037171a-040e-4b0e-a635-a0b500821338-iptables-alerter-script\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:47:37.420938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420914    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-kubelet\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.420938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420935    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-lib-modules\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.421120 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.420955    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee88ccb5-4a04-431a-8963-7b8c7393604a-konnectivity-ca\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:37.421120 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421017    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cnibin\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.421120 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421048    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.421120 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421071    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2037171a-040e-4b0e-a635-a0b500821338-host-slash\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:47:37.421120 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421086    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-etc-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.421120 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421118    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-bin\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.421315 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421138    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-config\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.421315 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421188    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.421315 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421212    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-host\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.421315 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421244    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-system-cni-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.421315 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421264    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtmk2\" (UniqueName: \"kubernetes.io/projected/2037171a-040e-4b0e-a635-a0b500821338-kube-api-access-dtmk2\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:47:37.421315 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421292    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-netd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.421315 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421311    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-cnibin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.421596 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421405    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-kubelet\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.421596 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421431    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-socket-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.421596 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421469    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:37.421596 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421494    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-systemd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.421596 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421537    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-env-overrides\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.421596 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421575    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-sys-fs\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.421860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421644    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-conf\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.421860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421686    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-tmp\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.421860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421711    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee88ccb5-4a04-431a-8963-7b8c7393604a-agent-certs\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:37.421860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421743    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-node-log\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.421860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421777    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:37.421860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421804    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b6ec759-f178-45cc-9c8e-7f5982c233dc-tmp\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.421860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421809    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-modprobe-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.421860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421831    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/325964e5-52a3-4672-bda2-ed01ffeee604-tmp-dir\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421868    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg7cc\" (UniqueName: \"kubernetes.io/projected/325964e5-52a3-4672-bda2-ed01ffeee604-kube-api-access-hg7cc\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421881    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/325964e5-52a3-4672-bda2-ed01ffeee604-tmp-dir\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421894    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-netns\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421933    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-run\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421958    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-device-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.421985    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.422027    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/325964e5-52a3-4672-bda2-ed01ffeee604-hosts-file\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.422077    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-multus\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.422108    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-netns\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.422130    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhjgx\" (UniqueName: \"kubernetes.io/projected/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kube-api-access-mhjgx\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.422152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.422150    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-kubelet-config\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:37.422582 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.422190    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-socket-dir-parent\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.422582 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.422239    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-multus-daemon-config\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.422582 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.422276    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-sys\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.439460 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.439437    4407 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 02:47:37.439565 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.439495    4407 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:37.439720 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.439706    4407 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 02:47:37.439771 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.439742    4407 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.us-west-1.compute.internal"
Nov 02 02:47:37.461443 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.461419    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.461443 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.461422    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 02:47:37.461443 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.461436    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.461596 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.461445    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 02:47:37.461596 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.461499    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 02:47:37.461971 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.461953    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cjmh9\""
Nov 02 02:47:37.461971 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.461963    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-nvmsr\""
Nov 02 02:47:37.462167 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.462146    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 02:47:37.462255 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.462178    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6md5s\""
Nov 02 02:47:37.462255 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.462184    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 02:47:37.462255 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.462243    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x92pp\""
Nov 02 02:47:37.462403 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.462299    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-pbbl9\""
Nov 02 02:47:37.462403 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.462387    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.462506 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.462492    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-multus-daemon-config\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.464031 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.464016    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9gkth\""
Nov 02 02:47:37.470560 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.470545    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.470707 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.470694    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ffe7d41-e27f-4373-8117-c49008015a88-cni-binary-copy\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.472907 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.472878    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-original-pull-secret\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:37.472948 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.472920    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee88ccb5-4a04-431a-8963-7b8c7393604a-agent-certs\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:37.495275 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.495254    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25h4m\" (UniqueName: \"kubernetes.io/projected/7ffe7d41-e27f-4373-8117-c49008015a88-kube-api-access-25h4m\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.495988 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.495970    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9znzz\" (UniqueName: \"kubernetes.io/projected/1c995359-9608-4568-947c-b9464cf3c4c0-kube-api-access-9znzz\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:37.496364 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.496349    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjzw\" (UniqueName: \"kubernetes.io/projected/7f3cbd14-2c73-4682-97d1-9beb1df2007e-kube-api-access-lsjzw\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.510997 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.510967    4407 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 02:47:37.522440 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522415    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.522509 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522442    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.522509 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522459    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/850126c8-146a-4442-88d9-37e8f668e76e-data-volume\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.522509 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522475    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-k8s-cni-cncf-io\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.522509 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522490    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-etc-kubernetes\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.522509 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522504    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522522    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522526    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-registration-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522544    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522568    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6grpn\" (UniqueName: \"kubernetes.io/projected/3b9f52ac-c454-4aa5-bb64-3bdae890695d-kube-api-access-6grpn\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522581    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-k8s-cni-cncf-io\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522595    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-etc-kubernetes\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522591    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522604    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/850126c8-146a-4442-88d9-37e8f668e76e-data-volume\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522648    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-var-lib-kubelet\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522671    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-registration-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522679    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-var-lib-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522707    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.522740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522728    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523147 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522747    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-var-lib-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.523147 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522748    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb902c0e-8074-4363-90c5-04eaa2c82fb3-host\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:47:37.523147 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522779    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysconfig\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523147 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522789    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb902c0e-8074-4363-90c5-04eaa2c82fb3-host\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:47:37.523147 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522794    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-systemd\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523147 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522830    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-systemd\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523147 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522855    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysconfig\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523147 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523024    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.523424 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523334    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.523424 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523387    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-var-lib-kubelet\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523491 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.522841    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.523491 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523456    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.523491 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523474    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-dbus\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:37.523589 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523491    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-bin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523589 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523505    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-hostroot\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523589 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523530    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-conf-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523589 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523555    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-system-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523589 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523568    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-dbus\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:37.523589 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523571    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-bin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523589 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523579    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-multus-certs\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523590    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523597    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-hostroot\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523607    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-kubernetes\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523615    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-conf-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523638    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-system-cni-dir\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523655    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-multus-certs\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523673    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-kubernetes\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523699    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-systemd-units\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523672    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-systemd-units\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523723    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-slash\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523737    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-ovn\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523753    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-kubelet\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523768    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-lib-modules\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523782    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cnibin\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523799    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523815    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2037171a-040e-4b0e-a635-a0b500821338-host-slash\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:47:37.523831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523818    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-ovn\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523836    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-kubelet\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523855    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-slash\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523870    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523860    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2037171a-040e-4b0e-a635-a0b500821338-host-slash\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523885    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-etc-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523903    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-bin\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523918    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523922    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-etc-openvswitch\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523877    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cnibin\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523877    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-lib-modules\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523933    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-bin\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523948    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523969    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.523979    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-host\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524004    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-system-cni-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524028    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f3cbd14-2c73-4682-97d1-9beb1df2007e-system-cni-dir\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524052    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-netd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524069    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/850126c8-146a-4442-88d9-37e8f668e76e-crio-socket\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524101    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-cni-netd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524128    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-host\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524147    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524171    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbxr\" (UniqueName: \"kubernetes.io/projected/850126c8-146a-4442-88d9-37e8f668e76e-kube-api-access-5fbxr\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524193    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9v8q\" (UniqueName: \"kubernetes.io/projected/149f4882-f029-488f-ba54-99d60c431352-kube-api-access-w9v8q\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524209    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-cnibin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524229    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-kubelet\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524254    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-kubelet\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524261    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-socket-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524264    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-cnibin\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524290    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524314    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-systemd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524334    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-socket-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.524330 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524339    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-sys-fs\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524371    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwd84\" (UniqueName: \"kubernetes.io/projected/6fbda56c-15a7-4c75-bd74-92a331b780e1-kube-api-access-xwd84\") pod \"migrator-66df5d496b-q8tl6\" (UID: \"6fbda56c-15a7-4c75-bd74-92a331b780e1\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524384    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-sys-fs\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524396    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-run-systemd\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524399    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-conf\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524491    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-sysctl-conf\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524538    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-node-log\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524870    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-node-log\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524906    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b9f52ac-c454-4aa5-bb64-3bdae890695d-config-volume\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524923    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-modprobe-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524940    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-netns\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524954    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-run\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524971    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-device-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.524998    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/325964e5-52a3-4672-bda2-ed01ffeee604-hosts-file\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525007    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-run-netns\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525022    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-multus\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525047    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-netns\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525050    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77c73be0-a65c-490e-9c4e-e4c15006f5c8-device-dir\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525052    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-run\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525052    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-etc-modprobe-d\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525061    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/325964e5-52a3-4672-bda2-ed01ffeee604-hosts-file\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525071    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-kubelet-config\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525057    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-host-var-lib-cni-multus\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525100    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/367d8f5b-1f4f-4d94-a9aa-7b007cd260a5-kubelet-config\") pod \"global-pull-secret-syncer-zs4kx\" (UID: \"367d8f5b-1f4f-4d94-a9aa-7b007cd260a5\") " pod="kube-system/global-pull-secret-syncer-zs4kx"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525104    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-host-run-netns\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525146    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-socket-dir-parent\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525195    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ffe7d41-e27f-4373-8117-c49008015a88-multus-socket-dir-parent\") pod \"multus-bvqdp\" (UID: \"7ffe7d41-e27f-4373-8117-c49008015a88\") " pod="openshift-multus/multus-bvqdp"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525213    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-sys\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525241    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b6ec759-f178-45cc-9c8e-7f5982c233dc-sys\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525241    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b9f52ac-c454-4aa5-bb64-3bdae890695d-tmp-dir\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525268    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/149f4882-f029-488f-ba54-99d60c431352-cert\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525294    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-log-socket\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525309    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b9f52ac-c454-4aa5-bb64-3bdae890695d-tmp-dir\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.525350    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9913c84-a163-4a5b-b169-cc80806bc03a-log-socket\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.561650 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.561576    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.561768 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.561723    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 02:47:37.561831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.561778    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.561831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.561785    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.561831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.561809    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ghkrt\""
Nov 02 02:47:37.561831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.561810    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 02:47:37.562029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562015    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 02:47:37.562085 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562041    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 02:47:37.562134 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562081    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.562134 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562129    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 02:47:37.562233 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562166    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 02:47:37.562233 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562173    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7jkxs\""
Nov 02 02:47:37.562233 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562185    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gdqg4\""
Nov 02 02:47:37.562233 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562212    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.562367 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562234    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.562367 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562282    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5mjxc\""
Nov 02 02:47:37.562367 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562301    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.562681 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562666    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.562747 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562694    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5v9mr\""
Nov 02 02:47:37.562747 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562732    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 02:47:37.562845 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562781    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cgnbb\""
Nov 02 02:47:37.562845 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562789    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.562845 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562830    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562874    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562874    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562739    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562918    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vvbct\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562924    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562920    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562922    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562919    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562964    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562939    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 02:47:37.562981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562936    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.562975    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563009    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563020    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-b8q6l\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563009    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563032    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-85n6h\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563041    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563051    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563057    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563066    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563097    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-wsjnf\""
Nov 02 02:47:37.563342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.563176    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 02:47:37.564656 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.564617    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhjgx\" (UniqueName: \"kubernetes.io/projected/77c73be0-a65c-490e-9c4e-e4c15006f5c8-kube-api-access-mhjgx\") pod \"aws-ebs-csi-driver-node-r6wjd\" (UID: \"77c73be0-a65c-490e-9c4e-e4c15006f5c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6wjd"
Nov 02 02:47:37.564737 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.564709    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5rmj\" (UniqueName: \"kubernetes.io/projected/6090e8b3-5746-45d5-9a52-1b592b54afaf-kube-api-access-h5rmj\") pod \"network-check-target-2h58d\" (UID: \"6090e8b3-5746-45d5-9a52-1b592b54afaf\") " pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:37.564774 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.564712    4407 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 02:47:37.564841 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.564830    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:38.064808018 +0000 UTC m=+1.860156562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : secret "metrics-daemon-secret" not found
Nov 02 02:47:37.565127 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.565114    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg7cc\" (UniqueName: \"kubernetes.io/projected/325964e5-52a3-4672-bda2-ed01ffeee604-kube-api-access-hg7cc\") pod \"node-resolver-z7p4b\" (UID: \"325964e5-52a3-4672-bda2-ed01ffeee604\") " pod="openshift-dns/node-resolver-z7p4b"
Nov 02 02:47:37.565158 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.565127    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97v74\" (UniqueName: \"kubernetes.io/projected/fb902c0e-8074-4363-90c5-04eaa2c82fb3-kube-api-access-97v74\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:47:37.565281 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.565268    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b9f52ac-c454-4aa5-bb64-3bdae890695d-config-volume\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.566491 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.566476    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/149f4882-f029-488f-ba54-99d60c431352-cert\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:37.570659 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.570642    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fb902c0e-8074-4363-90c5-04eaa2c82fb3-serviceca\") pod \"node-ca-mdddz\" (UID: \"fb902c0e-8074-4363-90c5-04eaa2c82fb3\") " pod="openshift-image-registry/node-ca-mdddz"
Nov 02 02:47:37.570761 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.570747    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9913c84-a163-4a5b-b169-cc80806bc03a-ovn-node-metrics-cert\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.570804 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.570790    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-script-lib\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.570834 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.570807    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f3cbd14-2c73-4682-97d1-9beb1df2007e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rm9b\" (UID: \"7f3cbd14-2c73-4682-97d1-9beb1df2007e\") " pod="openshift-multus/multus-additional-cni-plugins-2rm9b"
Nov 02 02:47:37.571012 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.570998    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwzpc\" (UniqueName: \"kubernetes.io/projected/8b6ec759-f178-45cc-9c8e-7f5982c233dc-kube-api-access-vwzpc\") pod \"tuned-q8k8l\" (UID: \"8b6ec759-f178-45cc-9c8e-7f5982c233dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q8k8l"
Nov 02 02:47:37.571177 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.571156    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee88ccb5-4a04-431a-8963-7b8c7393604a-konnectivity-ca\") pod \"konnectivity-agent-b5wvk\" (UID: \"ee88ccb5-4a04-431a-8963-7b8c7393604a\") " pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:37.571245 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.571230    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2037171a-040e-4b0e-a635-a0b500821338-iptables-alerter-script\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:47:37.571329 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.571314    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-ovnkube-config\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.571905 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.571887    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f9913c84-a163-4a5b-b169-cc80806bc03a-env-overrides\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.573244 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.573226    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtmk2\" (UniqueName: \"kubernetes.io/projected/2037171a-040e-4b0e-a635-a0b500821338-kube-api-access-dtmk2\") pod \"iptables-alerter-fbwhk\" (UID: \"2037171a-040e-4b0e-a635-a0b500821338\") " pod="openshift-network-operator/iptables-alerter-fbwhk"
Nov 02 02:47:37.573304 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.573239    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhqj\" (UniqueName: \"kubernetes.io/projected/f9913c84-a163-4a5b-b169-cc80806bc03a-kube-api-access-9nhqj\") pod \"ovnkube-node-4bpfv\" (UID: \"f9913c84-a163-4a5b-b169-cc80806bc03a\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:37.582215 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.582180    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6grpn\" (UniqueName: \"kubernetes.io/projected/3b9f52ac-c454-4aa5-bb64-3bdae890695d-kube-api-access-6grpn\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.582215 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.582190    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9v8q\" (UniqueName: \"kubernetes.io/projected/149f4882-f029-488f-ba54-99d60c431352-kube-api-access-w9v8q\") pod \"ingress-canary-n4jz2\" (UID: \"149f4882-f029-488f-ba54-99d60c431352\") " pod="openshift-ingress-canary/ingress-canary-n4jz2"
Nov 02 02:47:37.582404 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.582381    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbxr\" (UniqueName: \"kubernetes.io/projected/850126c8-146a-4442-88d9-37e8f668e76e-kube-api-access-5fbxr\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.582526 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.582510    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwd84\" (UniqueName: \"kubernetes.io/projected/6fbda56c-15a7-4c75-bd74-92a331b780e1-kube-api-access-xwd84\") pod \"migrator-66df5d496b-q8tl6\" (UID: \"6fbda56c-15a7-4c75-bd74-92a331b780e1\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:37.626542 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.626516    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.626695 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.626547    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/850126c8-146a-4442-88d9-37e8f668e76e-crio-socket\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.626695 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.626564    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.626695 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.626614    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:37.626695 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.626668    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/850126c8-146a-4442-88d9-37e8f668e76e-crio-socket\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:37.626695 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.626670    4407 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:37.626839 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.626717    4407 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:37.626839 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.626745    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:38.126731847 +0000 UTC m=+1.922080404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:37.626839 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.626682    4407 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 02:47:37.626839 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.626787    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:38.126781354 +0000 UTC m=+1.922129897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : configmap "kube-rbac-proxy" not found
Nov 02 02:47:37.626839 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:37.626798    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:38.126793393 +0000 UTC m=+1.922141937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:37.713299 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.713273    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6"
Nov 02 02:47:37.754215 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:37.754196    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 02:47:38.130701 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:38.130661    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:38.130898 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:38.130717    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:38.130898 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:38.130756    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:38.130898 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:38.130785    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:38.130898 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:38.130802    4407 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:38.130898 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:38.130805    4407 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:38.130898 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:38.130859    4407 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 02:47:38.130898 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:38.130894    4407 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 02:47:38.130898 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:38.130863    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:39.130849173 +0000 UTC m=+2.926197716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:38.131158 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:38.130928    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:39.130910851 +0000 UTC m=+2.926259395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:38.131158 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:38.130942    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:39.130935909 +0000 UTC m=+2.926284452 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : secret "metrics-daemon-secret" not found
Nov 02 02:47:38.131158 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:38.130953    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:39.130948235 +0000 UTC m=+2.926296778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : configmap "kube-rbac-proxy" not found
Nov 02 02:47:38.380414 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:38.380382    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6" event={"ID":"6fbda56c-15a7-4c75-bd74-92a331b780e1","Type":"ContainerStarted","Data":"5b381e938cb6a97e61e7854f32a8c58c4c368d844d8cfe0fde4f685b4e82c40d"}
Nov 02 02:47:39.138162 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:39.138122    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:39.138333 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:39.138179    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:39.138333 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:39.138213    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:39.138333 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:39.138243    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:39.138333 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:39.138278    4407 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:39.138333 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:39.138328    4407 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 02:47:39.138594 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:39.138337    4407 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 02:47:39.138594 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:39.138359    4407 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:39.138594 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:39.138344    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:41.138326337 +0000 UTC m=+4.933674898 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:39.138594 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:39.138416    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:41.138398944 +0000 UTC m=+4.933747486 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : secret "metrics-daemon-secret" not found
Nov 02 02:47:39.138594 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:39.138451    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:41.13842234 +0000 UTC m=+4.933770887 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : configmap "kube-rbac-proxy" not found
Nov 02 02:47:39.138594 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:39.138473    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:41.138464663 +0000 UTC m=+4.933813210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:39.310602 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:39.310573    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:39.776345 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:39.776318    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:40.387091 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:40.387062    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q8tl6" event={"ID":"6fbda56c-15a7-4c75-bd74-92a331b780e1","Type":"ContainerStarted","Data":"51a8b0ed5ca85d6e869345538843f06a94037463de16201e44ce7da99622a73e"}
Nov 02 02:47:41.149923 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:41.149866    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:41.149949    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:41.149969    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:41.149990    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:41.150034    4407 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:41.150111    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:45.150090006 +0000 UTC m=+8.945438567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:41.150116    4407 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:41.150120    4407 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:41.150175    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:45.150158561 +0000 UTC m=+8.945507124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:41.150451 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:41.150234    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:45.150199256 +0000 UTC m=+8.945547802 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : secret "metrics-daemon-secret" not found
Nov 02 02:47:41.150762 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:41.150602    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/850126c8-146a-4442-88d9-37e8f668e76e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:41.817748 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:41.817718    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:43.399237 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:43.399212    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:47:43.399615 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:43.399394    4407 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 02:47:44.071784 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:44.071746    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:44.854975 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:44.854939    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:45.105249 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:45.105166    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z7p4b_325964e5-52a3-4672-bda2-ed01ffeee604/dns-node-resolver/0.log"
Nov 02 02:47:45.175681 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:45.175605    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:45.175877 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:45.175697    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:45.175877 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:45.175729    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:45.175877 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:45.175781    4407 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 02:47:45.175877 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:45.175819    4407 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 02:47:45.175877 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:45.175820    4407 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:45.175877 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:45.175870    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls podName:3b9f52ac-c454-4aa5-bb64-3bdae890695d nodeName:}" failed. No retries permitted until 2026-11-02 02:47:53.17585046 +0000 UTC m=+16.971199017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls") pod "dns-default-st5ft" (UID: "3b9f52ac-c454-4aa5-bb64-3bdae890695d") : secret "dns-default-metrics-tls" not found
Nov 02 02:47:45.176105 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:45.175889    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs podName:1c995359-9608-4568-947c-b9464cf3c4c0 nodeName:}" failed. No retries permitted until 2026-11-02 02:47:53.175880906 +0000 UTC m=+16.971229450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs") pod "network-metrics-daemon-4xrlc" (UID: "1c995359-9608-4568-947c-b9464cf3c4c0") : secret "metrics-daemon-secret" not found
Nov 02 02:47:45.176105 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:47:45.175905    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls podName:850126c8-146a-4442-88d9-37e8f668e76e nodeName:}" failed. No retries permitted until 2026-11-02 02:47:53.175897864 +0000 UTC m=+16.971246408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vzzk7" (UID: "850126c8-146a-4442-88d9-37e8f668e76e") : secret "insights-runtime-extractor-tls" not found
Nov 02 02:47:45.287291 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:45.287255    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:45.448522 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:45.448446    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:45.739239 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:45.739149    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:45.922848 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:45.922813    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:46.104686 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:46.104656    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mdddz_fb902c0e-8074-4363-90c5-04eaa2c82fb3/node-ca/0.log"
Nov 02 02:47:46.505137 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:46.505109    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n4jz2_149f4882-f029-488f-ba54-99d60c431352/serve-healthcheck-canary/0.log"
Nov 02 02:47:46.515777 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:46.515749    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:46.750987 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:46.750962    4407 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-112.us-west-1.compute.internal" event="NodeReady"
Nov 02 02:47:46.907077 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:46.906990    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:47.105769 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:47.105741    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-q8tl6_6fbda56c-15a7-4c75-bd74-92a331b780e1/migrator/0.log"
Nov 02 02:47:47.121308 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:47.121278    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:47.313935 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:47.313906    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:47.501898 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:47.501866    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:47:47.715050 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:47.714973    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:48.507265 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:48.507230    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:48.908869 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:48.908799    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:49.118403 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:49.118367    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:49.314855 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:49.312712    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:49.707554 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:49.707475    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:50.225773 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:50.225746    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b5wvk"
Nov 02 02:47:50.511061 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:50.511039    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:50.911256 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:50.911171    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:51.110866 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:51.110838    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:51.315611 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:51.315583    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:51.617387 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:51.617313    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:51.777403 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:51.777375    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:51.920245 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:51.920167    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:52.070247 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:52.070215    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:52.513840 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:52.513808    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:52.667731 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:52.667702    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:52.825788 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:52.825704    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:53.111124 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.111046    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:53.229225 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.229194    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:53.229407 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.229235    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:53.229407 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.229269    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:53.229723 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.229705    4407 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 02:47:53.232005 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.231982    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b9f52ac-c454-4aa5-bb64-3bdae890695d-metrics-tls\") pod \"dns-default-st5ft\" (UID: \"3b9f52ac-c454-4aa5-bb64-3bdae890695d\") " pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:53.232091 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.232032    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c995359-9608-4568-947c-b9464cf3c4c0-metrics-certs\") pod \"network-metrics-daemon-4xrlc\" (UID: \"1c995359-9608-4568-947c-b9464cf3c4c0\") " pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:53.232129 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.232087    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/850126c8-146a-4442-88d9-37e8f668e76e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vzzk7\" (UID: \"850126c8-146a-4442-88d9-37e8f668e76e\") " pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:53.308304 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.308275    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xrlc"
Nov 02 02:47:53.309365 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.309332    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:53.313175 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.313148    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:53.314335 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.314245    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vzzk7"
Nov 02 02:47:53.506551 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.506517    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:53.659204 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.659129    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:53.677587 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.677566    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4xrlc"]
Nov 02 02:47:53.720565 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:53.720534    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c995359_9608_4568_947c_b9464cf3c4c0.slice/crio-262cd65988fde0849dd996c08223cfaa82a223b122a3e116374621513a9ba5ee WatchSource:0}: Error finding container 262cd65988fde0849dd996c08223cfaa82a223b122a3e116374621513a9ba5ee: Status 404 returned error can't find the container with id 262cd65988fde0849dd996c08223cfaa82a223b122a3e116374621513a9ba5ee
Nov 02 02:47:53.729541 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.729517    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-st5ft"]
Nov 02 02:47:53.732018 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:53.731996    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9f52ac_c454_4aa5_bb64_3bdae890695d.slice/crio-d7cb293eb91c58124306cee1d9daafe284468dfaeaf59a0ff5deb48a5ab4f772 WatchSource:0}: Error finding container d7cb293eb91c58124306cee1d9daafe284468dfaeaf59a0ff5deb48a5ab4f772: Status 404 returned error can't find the container with id d7cb293eb91c58124306cee1d9daafe284468dfaeaf59a0ff5deb48a5ab4f772
Nov 02 02:47:53.737525 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.737507    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vzzk7"]
Nov 02 02:47:53.739695 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:53.739670    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850126c8_146a_4442_88d9_37e8f668e76e.slice/crio-d3faf1ae7acd1b4bb28a515fd869daddd12e58f48b55a9c4aa8bc5652e94acc5 WatchSource:0}: Error finding container d3faf1ae7acd1b4bb28a515fd869daddd12e58f48b55a9c4aa8bc5652e94acc5: Status 404 returned error can't find the container with id d3faf1ae7acd1b4bb28a515fd869daddd12e58f48b55a9c4aa8bc5652e94acc5
Nov 02 02:47:53.918876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:53.918792    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:54.062542 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:54.062505    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:54.418742 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:54.418707    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xrlc" event={"ID":"1c995359-9608-4568-947c-b9464cf3c4c0","Type":"ContainerStarted","Data":"262cd65988fde0849dd996c08223cfaa82a223b122a3e116374621513a9ba5ee"}
Nov 02 02:47:54.420089 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:54.420073    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vzzk7" event={"ID":"850126c8-146a-4442-88d9-37e8f668e76e","Type":"ContainerStarted","Data":"e6506131b94ef8f1e18ce2bd36e1c5480e470316914caf8cb685489af2f2c50d"}
Nov 02 02:47:54.420151 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:54.420112    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vzzk7" event={"ID":"850126c8-146a-4442-88d9-37e8f668e76e","Type":"ContainerStarted","Data":"d3faf1ae7acd1b4bb28a515fd869daddd12e58f48b55a9c4aa8bc5652e94acc5"}
Nov 02 02:47:54.421214 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:54.421193    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-st5ft" event={"ID":"3b9f52ac-c454-4aa5-bb64-3bdae890695d","Type":"ContainerStarted","Data":"d7cb293eb91c58124306cee1d9daafe284468dfaeaf59a0ff5deb48a5ab4f772"}
Nov 02 02:47:54.516586 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:54.516554    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:54.670269 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:54.670191    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:54.822667 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:54.822613    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:55.110228 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.110194    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:55.314802 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.314705    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:55.427088 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.427052    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vzzk7" event={"ID":"850126c8-146a-4442-88d9-37e8f668e76e","Type":"ContainerStarted","Data":"c1324b423942b9337d403104fdc4eea30b53a9d073cdfafe6f71f10b44de2d05"}
Nov 02 02:47:55.428697 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.428662    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xrlc" event={"ID":"1c995359-9608-4568-947c-b9464cf3c4c0","Type":"ContainerStarted","Data":"973d92e72a51858167424b44c679291e66c78adfc88f083eeeb22ea0fecd4407"}
Nov 02 02:47:55.673210 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.673140    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6654798c4f-2sp4f"]
Nov 02 02:47:55.675408 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.675388    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.741128 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.741062    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:55.747817 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.747787    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-installation-pull-secrets\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.747991 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.747831    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-tls\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.747991 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.747931    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-image-registry-private-configuration\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.747991 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.747965    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-trusted-ca\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.748136 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.747998    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-bound-sa-token\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.748136 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.748110    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxcl2\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-kube-api-access-wxcl2\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.748216 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.748142    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-certificates\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.748216 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.748169    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8d445b-011f-4a47-987e-8d54a0a11cc7-ca-trust-extracted\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.835867 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.835816    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 02 02:47:55.835867 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.835831    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 02 02:47:55.835867 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.835828    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hws9m\""
Nov 02 02:47:55.836172 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.835884    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 02 02:47:55.836172 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.835928    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 02 02:47:55.836582 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.836563    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6654798c4f-2sp4f"]
Nov 02 02:47:55.849367 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.849337    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-tls\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.849574 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.849385    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-image-registry-private-configuration\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.849574 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.849409    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-trusted-ca\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.849574 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.849440    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-bound-sa-token\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.849574 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.849474    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxcl2\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-kube-api-access-wxcl2\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.849574 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.849500    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-certificates\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.849574 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.849537    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8d445b-011f-4a47-987e-8d54a0a11cc7-ca-trust-extracted\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.850649 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.849607    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-installation-pull-secrets\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.855563 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.855534    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-trusted-ca\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.855955 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.855931    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-tls\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.856456 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.856435    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-certificates\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.856571 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.856552    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8d445b-011f-4a47-987e-8d54a0a11cc7-ca-trust-extracted\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.859339 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.859311    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-image-registry-private-configuration\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.861243 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.861218    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-installation-pull-secrets\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.912607 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.912570    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:55.913586 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.913558    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-bound-sa-token\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.914717 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.914687    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxcl2\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-kube-api-access-wxcl2\") pod \"image-registry-6654798c4f-2sp4f\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") " pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:55.946134 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.946062    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cf856d97b-94pxg"]
Nov 02 02:47:55.948490 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.948466    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:55.988000 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:55.987983    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:56.015199 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.015165    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cf856d97b-94pxg"]
Nov 02 02:47:56.055945 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.055915    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd476338-bec8-4a9a-a611-918ee8b23e8d-installation-pull-secrets\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.056059 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.055973    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-bound-sa-token\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.056059 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.056018    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd476338-bec8-4a9a-a611-918ee8b23e8d-ca-trust-extracted\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.056059 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.056047    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd476338-bec8-4a9a-a611-918ee8b23e8d-registry-certificates\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.056156 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.056084    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd476338-bec8-4a9a-a611-918ee8b23e8d-image-registry-private-configuration\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.056156 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.056114    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd476338-bec8-4a9a-a611-918ee8b23e8d-trusted-ca\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.056156 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.056138    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s4wx\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-kube-api-access-6s4wx\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.056240 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.056182    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-registry-tls\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.077799 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.077756    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:56.156907 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.156884    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-bound-sa-token\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157027 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.156926    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd476338-bec8-4a9a-a611-918ee8b23e8d-ca-trust-extracted\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157027 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.156949    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd476338-bec8-4a9a-a611-918ee8b23e8d-registry-certificates\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157027 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.156986    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd476338-bec8-4a9a-a611-918ee8b23e8d-image-registry-private-configuration\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157027 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.157014    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd476338-bec8-4a9a-a611-918ee8b23e8d-trusted-ca\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157217 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.157038    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s4wx\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-kube-api-access-6s4wx\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157217 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.157074    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-registry-tls\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157217 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.157112    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd476338-bec8-4a9a-a611-918ee8b23e8d-installation-pull-secrets\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157427 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.157288    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd476338-bec8-4a9a-a611-918ee8b23e8d-ca-trust-extracted\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.157959 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.157926    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd476338-bec8-4a9a-a611-918ee8b23e8d-registry-certificates\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.158077 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.158053    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd476338-bec8-4a9a-a611-918ee8b23e8d-trusted-ca\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.159987 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.159965    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd476338-bec8-4a9a-a611-918ee8b23e8d-installation-pull-secrets\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.160092 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.159976    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-registry-tls\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.160159 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.160108    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd476338-bec8-4a9a-a611-918ee8b23e8d-image-registry-private-configuration\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.215485 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.215408    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-bound-sa-token\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.215742 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.215721    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s4wx\" (UniqueName: \"kubernetes.io/projected/fd476338-bec8-4a9a-a611-918ee8b23e8d-kube-api-access-6s4wx\") pod \"image-registry-6cf856d97b-94pxg\" (UID: \"fd476338-bec8-4a9a-a611-918ee8b23e8d\") " pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.259447 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.259414    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:56.287740 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.287710    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:56.416256 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.416224    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6654798c4f-2sp4f"]
Nov 02 02:47:56.419399 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:56.419367    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8d445b_011f_4a47_987e_8d54a0a11cc7.slice/crio-659623cdd3f7a316d09e10c398c92f060fee5aa75ec2a089464bc486abe759fd WatchSource:0}: Error finding container 659623cdd3f7a316d09e10c398c92f060fee5aa75ec2a089464bc486abe759fd: Status 404 returned error can't find the container with id 659623cdd3f7a316d09e10c398c92f060fee5aa75ec2a089464bc486abe759fd
Nov 02 02:47:56.441308 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.441281    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:47:56.442396 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.441944    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xrlc" event={"ID":"1c995359-9608-4568-947c-b9464cf3c4c0","Type":"ContainerStarted","Data":"c2a6601686ab7090ce1179c891e959d8c2bba7409ff17aa964b4a98a25e8827e"}
Nov 02 02:47:56.443817 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.443791    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-st5ft" event={"ID":"3b9f52ac-c454-4aa5-bb64-3bdae890695d","Type":"ContainerStarted","Data":"77bb7fe1e84818fbbc2c4328543bf739c260ea44770a8c32d049888a73384d49"}
Nov 02 02:47:56.443919 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.443837    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-st5ft" event={"ID":"3b9f52ac-c454-4aa5-bb64-3bdae890695d","Type":"ContainerStarted","Data":"6bbd0f57c6dcc3a40f043f295c2c2736aee1551df60443dc16728a32e0a63dfa"}
Nov 02 02:47:56.443919 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.443873    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-st5ft"
Nov 02 02:47:56.445144 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.445122    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f" event={"ID":"9b8d445b-011f-4a47-987e-8d54a0a11cc7","Type":"ContainerStarted","Data":"659623cdd3f7a316d09e10c398c92f060fee5aa75ec2a089464bc486abe759fd"}
Nov 02 02:47:56.635603 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:56.635564    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cf856d97b-94pxg"]
Nov 02 02:47:56.638719 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:47:56.638689    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd476338_bec8_4a9a_a611_918ee8b23e8d.slice/crio-0c9509a0a6110d32d2f982aed6493d2551e11ba801d165d9913e97dd8ee1a03f WatchSource:0}: Error finding container 0c9509a0a6110d32d2f982aed6493d2551e11ba801d165d9913e97dd8ee1a03f: Status 404 returned error can't find the container with id 0c9509a0a6110d32d2f982aed6493d2551e11ba801d165d9913e97dd8ee1a03f
Nov 02 02:47:57.450735 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:57.450616    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vzzk7" event={"ID":"850126c8-146a-4442-88d9-37e8f668e76e","Type":"ContainerStarted","Data":"9a2df61d3962f3fc2e0f1dbe5bc79c066593f425937534fb2dbe3dece4082247"}
Nov 02 02:47:57.452039 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:57.452016    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f" event={"ID":"9b8d445b-011f-4a47-987e-8d54a0a11cc7","Type":"ContainerStarted","Data":"37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba"}
Nov 02 02:47:57.452135 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:57.452073    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:47:57.453441 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:57.453413    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf856d97b-94pxg" event={"ID":"fd476338-bec8-4a9a-a611-918ee8b23e8d","Type":"ContainerStarted","Data":"63385b77f76bbffea179c15390766dec0fed9888314330f208a75d086e288b8b"}
Nov 02 02:47:57.453571 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:57.453454    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf856d97b-94pxg" event={"ID":"fd476338-bec8-4a9a-a611-918ee8b23e8d","Type":"ContainerStarted","Data":"0c9509a0a6110d32d2f982aed6493d2551e11ba801d165d9913e97dd8ee1a03f"}
Nov 02 02:47:57.453571 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:57.453524    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:47:57.682426 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:57.682363    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f" podStartSLOduration=2.682348307 podStartE2EDuration="2.682348307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:57.681942953 +0000 UTC m=+21.477291556" watchObservedRunningTime="2026-11-02 02:47:57.682348307 +0000 UTC m=+21.477696863"
Nov 02 02:47:57.799658 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:57.799589    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cf856d97b-94pxg" podStartSLOduration=2.799577581 podStartE2EDuration="2.799577581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:47:57.799002617 +0000 UTC m=+21.594351185" watchObservedRunningTime="2026-11-02 02:47:57.799577581 +0000 UTC m=+21.594926145"
Nov 02 02:47:59.885637 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:47:59.885587    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:00.036260 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:00.036222    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:00.186962 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:00.186894    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:00.699914 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:00.699875    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:00.861738 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:00.861702    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:01.031814 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:01.031775    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:01.176331 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:01.176300    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:02.928221 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:02.928194    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bpfv"
Nov 02 02:48:04.224249 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:04.224220    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:04.414107 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:04.414069    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:04.568199 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:04.568166    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:04.727700 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:04.727668    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:05.188259 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:48:05.188221    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:48:06.455339 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:06.455307    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-st5ft"
Nov 02 02:48:07.505210 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:07.505177    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2h58d"
Nov 02 02:48:09.934426 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:09.934396    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:10.625206 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:10.625169    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:10.852518 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:10.852474    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:11.087419 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.087389    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:11.272005 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.271973    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:11.381472 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.381399    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m8dft"]
Nov 02 02:48:11.384674 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.384655    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.451920 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.451883    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-root\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.451920 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.451923    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-textfile\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.452114 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.451941    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.452114 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.452090    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-sys\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.452182 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.452124    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.452214 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.452183    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-tls\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.452214 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.452204    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eec203a-7c01-43d3-ae10-0c6e423f73e4-metrics-client-ca\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.452276 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.452220    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp2mv\" (UniqueName: \"kubernetes.io/projected/0eec203a-7c01-43d3-ae10-0c6e423f73e4-kube-api-access-sp2mv\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.452276 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.452258    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-wtmp\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.459291 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.459270    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 02 02:48:11.459353 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.459291    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gtx4f\""
Nov 02 02:48:11.462903 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.462881    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:11.489573 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.489556    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 02:48:11.489713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.489563    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 02 02:48:11.489713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.489608    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 02:48:11.489713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.489658    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 02 02:48:11.489831 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.489808    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 02 02:48:11.553430 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553394    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp2mv\" (UniqueName: \"kubernetes.io/projected/0eec203a-7c01-43d3-ae10-0c6e423f73e4-kube-api-access-sp2mv\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553430 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553429    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-wtmp\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553457    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-root\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553477    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-textfile\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553493    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553546    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-root\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553596    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-sys\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553644    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-wtmp\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553660    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553698    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-tls\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.553713 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553706    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eec203a-7c01-43d3-ae10-0c6e423f73e4-sys\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.554069 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553725    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eec203a-7c01-43d3-ae10-0c6e423f73e4-metrics-client-ca\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.554069 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.553899    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-textfile\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.554262 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.554242    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eec203a-7c01-43d3-ae10-0c6e423f73e4-metrics-client-ca\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.554310 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.554243    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.555933 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.555899    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.556049 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.556020    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0eec203a-7c01-43d3-ae10-0c6e423f73e4-node-exporter-tls\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.610851 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.610825    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp2mv\" (UniqueName: \"kubernetes.io/projected/0eec203a-7c01-43d3-ae10-0c6e423f73e4-kube-api-access-sp2mv\") pod \"node-exporter-m8dft\" (UID: \"0eec203a-7c01-43d3-ae10-0c6e423f73e4\") " pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.627727 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.627696    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:11.695884 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.695811    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m8dft"
Nov 02 02:48:11.707409 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:48:11.707378    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eec203a_7c01_43d3_ae10_0c6e423f73e4.slice/crio-8d53f855a99f066f33a2a776f8c8c75939d64d2c6aba45b7759325f5faa61da7 WatchSource:0}: Error finding container 8d53f855a99f066f33a2a776f8c8c75939d64d2c6aba45b7759325f5faa61da7: Status 404 returned error can't find the container with id 8d53f855a99f066f33a2a776f8c8c75939d64d2c6aba45b7759325f5faa61da7
Nov 02 02:48:11.789809 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:11.789779    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:12.031603 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:12.031571    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:12.505058 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:12.505019    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8dft" event={"ID":"0eec203a-7c01-43d3-ae10-0c6e423f73e4","Type":"ContainerStarted","Data":"8d53f855a99f066f33a2a776f8c8c75939d64d2c6aba45b7759325f5faa61da7"}
Nov 02 02:48:12.564283 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:12.564246    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:12.732249 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:12.732214    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:13.434641 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.434595    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:13.493706 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.493680    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 02:48:13.495887 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.495869    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.508557 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.508535    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 02:48:13.510588 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.510563    4407 generic.go:356] "Generic (PLEG): container finished" podID="0eec203a-7c01-43d3-ae10-0c6e423f73e4" containerID="4fa54fa5392a59f5bd7fc345bc9d2a6e5c8cd54f581c935a248827d60a3682fa" exitCode=0
Nov 02 02:48:13.510718 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.510640    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8dft" event={"ID":"0eec203a-7c01-43d3-ae10-0c6e423f73e4","Type":"ContainerDied","Data":"4fa54fa5392a59f5bd7fc345bc9d2a6e5c8cd54f581c935a248827d60a3682fa"}
Nov 02 02:48:13.557197 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.557174    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 02 02:48:13.558063 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.558045    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 02 02:48:13.558296 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.558277    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 02 02:48:13.558367 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.558299    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 02 02:48:13.558367 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.558311    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 02 02:48:13.558367 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.558313    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 02 02:48:13.558367 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.558298    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 02 02:48:13.558583 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.558406    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 02 02:48:13.558583 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.558443    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-69rrk\""
Nov 02 02:48:13.568852 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.568829    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.568932 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.568863    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d3eea3a-2a6c-4acf-b811-e261852d0a58-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.568994 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.568977    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569121 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569101    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g849l\" (UniqueName: \"kubernetes.io/projected/9d3eea3a-2a6c-4acf-b811-e261852d0a58-kube-api-access-g849l\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569178 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569142    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569178 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569170    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-web-config\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569273 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569216    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d3eea3a-2a6c-4acf-b811-e261852d0a58-config-out\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569273 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569239    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569273 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569269    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d3eea3a-2a6c-4acf-b811-e261852d0a58-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569429 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569313    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d3eea3a-2a6c-4acf-b811-e261852d0a58-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569429 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569384    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569542 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569513    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.569595 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.569543    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d3eea3a-2a6c-4acf-b811-e261852d0a58-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.670851 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.670820    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d3eea3a-2a6c-4acf-b811-e261852d0a58-config-out\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.670858    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.670886    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d3eea3a-2a6c-4acf-b811-e261852d0a58-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.670908    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d3eea3a-2a6c-4acf-b811-e261852d0a58-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.670932    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.670968    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671029 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.670989    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d3eea3a-2a6c-4acf-b811-e261852d0a58-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671285 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.671038    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671285 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.671119    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d3eea3a-2a6c-4acf-b811-e261852d0a58-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671285 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.671167    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671285 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.671202    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g849l\" (UniqueName: \"kubernetes.io/projected/9d3eea3a-2a6c-4acf-b811-e261852d0a58-kube-api-access-g849l\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671285 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.671232    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671285 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.671244    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d3eea3a-2a6c-4acf-b811-e261852d0a58-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.671285 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.671258    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-web-config\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.672395 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.672367    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d3eea3a-2a6c-4acf-b811-e261852d0a58-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.674023 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.673970    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.674226 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.674209    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.674388 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.674367    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.674841 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.674802    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d3eea3a-2a6c-4acf-b811-e261852d0a58-config-out\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.674841 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.674812    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d3eea3a-2a6c-4acf-b811-e261852d0a58-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.675020 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.675001    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.675080 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.675061    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.675709 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.675691    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-web-config\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.676382 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.676354    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d3eea3a-2a6c-4acf-b811-e261852d0a58-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.680818 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.680795    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 02 02:48:13.682057 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.682036    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d3eea3a-2a6c-4acf-b811-e261852d0a58-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.729480 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.729459    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g849l\" (UniqueName: \"kubernetes.io/projected/9d3eea3a-2a6c-4acf-b811-e261852d0a58-kube-api-access-g849l\") pod \"alertmanager-main-0\" (UID: \"9d3eea3a-2a6c-4acf-b811-e261852d0a58\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:13.811279 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:13.811198    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 02 02:48:14.231587 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:14.231562    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 02:48:14.233789 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:48:14.233759    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3eea3a_2a6c_4acf_b811_e261852d0a58.slice/crio-e83884f223d28425db70c4cd4206d4cc7e2eba692ff627c16dc63adfbd4f4c86 WatchSource:0}: Error finding container e83884f223d28425db70c4cd4206d4cc7e2eba692ff627c16dc63adfbd4f4c86: Status 404 returned error can't find the container with id e83884f223d28425db70c4cd4206d4cc7e2eba692ff627c16dc63adfbd4f4c86
Nov 02 02:48:14.515672 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:14.515637    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8dft" event={"ID":"0eec203a-7c01-43d3-ae10-0c6e423f73e4","Type":"ContainerStarted","Data":"0fbf7f9000340aca65be8fdf4db4f5378bf3606eb68d8cf9eb051944dd05c4c6"}
Nov 02 02:48:14.516119 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:14.515701    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8dft" event={"ID":"0eec203a-7c01-43d3-ae10-0c6e423f73e4","Type":"ContainerStarted","Data":"f8996c2542b105e4df41991bf01194283260b9448712f034dff7395e3e1daff4"}
Nov 02 02:48:14.516956 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:14.516932    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d3eea3a-2a6c-4acf-b811-e261852d0a58","Type":"ContainerStarted","Data":"e83884f223d28425db70c4cd4206d4cc7e2eba692ff627c16dc63adfbd4f4c86"}
Nov 02 02:48:14.632711 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:14.632660    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m8dft" podStartSLOduration=2.032644657 podStartE2EDuration="3.632647306s" totalImagesPullingTime="1.5248309s" totalInitContainerRuntime="75.171749ms" isStatefulPod=true podCreationTimestamp="2026-11-02 02:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:48:14.631742312 +0000 UTC m=+38.427090876" watchObservedRunningTime="2026-11-02 02:48:14.632647306 +0000 UTC m=+38.427995871"
Nov 02 02:48:15.328160 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:15.328131    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:15.516616 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:15.516575    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:15.709647 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:15.709602    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:15.876574 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:15.876543    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:16.042472 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.042436    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:16.299557 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.299477    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:16.524957 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.524924    4407 generic.go:356] "Generic (PLEG): container finished" podID="9d3eea3a-2a6c-4acf-b811-e261852d0a58" containerID="e2a06c86792a9de2e0800ffb932595fdd21c305626f856d5c8438c07c1c736d0" exitCode=0
Nov 02 02:48:16.525459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.525001    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d3eea3a-2a6c-4acf-b811-e261852d0a58","Type":"ContainerDied","Data":"e2a06c86792a9de2e0800ffb932595fdd21c305626f856d5c8438c07c1c736d0"}
Nov 02 02:48:16.908061 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.908027    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"]
Nov 02 02:48:16.910000 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.909986    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:16.963222 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.963196    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 02 02:48:16.963363 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.963321    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 02 02:48:16.963418 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.963387    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 02 02:48:16.963467 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.963404    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 02 02:48:16.963506 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.963482    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 02 02:48:16.963736 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.963719    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-t8vpm\""
Nov 02 02:48:16.996256 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.996223    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:16.996393 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.996271    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-serving-certs-ca-bundle\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:16.996393 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.996296    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg9zx\" (UniqueName: \"kubernetes.io/projected/e1ebd107-87fa-4417-98bc-68c9bcb531e6-kube-api-access-vg9zx\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:16.996393 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.996386    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-secret-telemeter-client\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:16.996530 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.996409    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:16.996530 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.996475    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-telemeter-client-tls\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:16.996530 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.996499    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-metrics-client-ca\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:16.996530 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:16.996521    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-federate-client-tls\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.014789 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.014764    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 02 02:48:17.015600 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.015584    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"]
Nov 02 02:48:17.097131 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.097101    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-serving-certs-ca-bundle\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.097131 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.097133    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg9zx\" (UniqueName: \"kubernetes.io/projected/e1ebd107-87fa-4417-98bc-68c9bcb531e6-kube-api-access-vg9zx\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.097445 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.097153    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-secret-telemeter-client\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.097445 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.097171    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.097445 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.097218    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-telemeter-client-tls\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.097445 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.097251    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-metrics-client-ca\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.097445 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.097284    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-federate-client-tls\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.097445 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.097319    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.098074 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.098029    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-serving-certs-ca-bundle\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.098074 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.098056    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-metrics-client-ca\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.098348 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.098328    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ebd107-87fa-4417-98bc-68c9bcb531e6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.100290 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.100263    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.100417 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.100396    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-secret-telemeter-client\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.100516 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.100496    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-telemeter-client-tls\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.100710 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.100692    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e1ebd107-87fa-4417-98bc-68c9bcb531e6-federate-client-tls\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.155748 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.155720    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg9zx\" (UniqueName: \"kubernetes.io/projected/e1ebd107-87fa-4417-98bc-68c9bcb531e6-kube-api-access-vg9zx\") pod \"telemeter-client-5bddbb5fbf-rdj2j\" (UID: \"e1ebd107-87fa-4417-98bc-68c9bcb531e6\") " pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.221244 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.221162    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"
Nov 02 02:48:17.688829 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.688797    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j"]
Nov 02 02:48:17.727676 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:48:17.727593    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ebd107_87fa_4417_98bc_68c9bcb531e6.slice/crio-4fa1d7dfe1fc9a0a61f9d29f1930d72243bc9a5a2d2ded19805d4777d502899e WatchSource:0}: Error finding container 4fa1d7dfe1fc9a0a61f9d29f1930d72243bc9a5a2d2ded19805d4777d502899e: Status 404 returned error can't find the container with id 4fa1d7dfe1fc9a0a61f9d29f1930d72243bc9a5a2d2ded19805d4777d502899e
Nov 02 02:48:17.906847 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:17.906789    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:18.086664 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.086610    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:18.463096 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.463074    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:48:18.463981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.463963    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cf856d97b-94pxg"
Nov 02 02:48:18.534743 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.534706    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d3eea3a-2a6c-4acf-b811-e261852d0a58","Type":"ContainerStarted","Data":"7239a66ef94585b15190a1f64c4cf990175455bfa42c3c2b58152a62174c11af"}
Nov 02 02:48:18.534931 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.534791    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d3eea3a-2a6c-4acf-b811-e261852d0a58","Type":"ContainerStarted","Data":"b51b5ddca9970a14cd3af1d087a0910783a5a34b588326d849857ccd5407de04"}
Nov 02 02:48:18.534931 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.534837    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d3eea3a-2a6c-4acf-b811-e261852d0a58","Type":"ContainerStarted","Data":"efa9d2c87fbccbda32e2b2179ca284b03c5d07799aa970b51e921e77a5558415"}
Nov 02 02:48:18.534931 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.534875    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d3eea3a-2a6c-4acf-b811-e261852d0a58","Type":"ContainerStarted","Data":"e8b7f9ed1b4bcce9fe168c4309e6b81def685329cdae8271c8f484bdd6ff0f19"}
Nov 02 02:48:18.534931 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.534912    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d3eea3a-2a6c-4acf-b811-e261852d0a58","Type":"ContainerStarted","Data":"c99fb349f6ebd906396af5f4e57cfa523ada7a16cab36ed0c51ae3308ee37c7e"}
Nov 02 02:48:18.535967 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.535948    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j" event={"ID":"e1ebd107-87fa-4417-98bc-68c9bcb531e6","Type":"ContainerStarted","Data":"4fa1d7dfe1fc9a0a61f9d29f1930d72243bc9a5a2d2ded19805d4777d502899e"}
Nov 02 02:48:18.725834 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.725763    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6654798c4f-2sp4f"]
Nov 02 02:48:18.851471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.851434    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:18.999328 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:18.999296    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:19.148361 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:19.148333    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:20.333970 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.333936    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:20.393525 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.393496    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 02:48:20.395820 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.395805    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.451222 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.451195    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 02:48:20.500609 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.500581    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 02 02:48:20.501403 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.501388    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 02 02:48:20.501464 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.501447    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 02 02:48:20.501507 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.501473    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 02 02:48:20.501555 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.501504    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-43opnl065ml03\""
Nov 02 02:48:20.501986 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.501972    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9nqbs\""
Nov 02 02:48:20.502386 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.502372    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 02 02:48:20.529303 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529282    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529358 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529314    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529358 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529341    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529460 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529358    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529460 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529388    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529539 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529456    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529539 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529481    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529539 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529500    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529662 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529548    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckc5x\" (UniqueName: \"kubernetes.io/projected/b9f9767f-d787-4e76-bea2-26669d01a593-kube-api-access-ckc5x\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529662 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529605    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9f9767f-d787-4e76-bea2-26669d01a593-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529662 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529650    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529774 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529676    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529774 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529740    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-config\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529774 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529760    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9f9767f-d787-4e76-bea2-26669d01a593-config-out\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529875 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529778    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529875 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529796    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529875 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529835    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.529875 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529864    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.530005 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529882    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.530005 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.529903    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-web-config\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.546098 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.546075    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d3eea3a-2a6c-4acf-b811-e261852d0a58","Type":"ContainerStarted","Data":"299cf584afdd0e2dbafc72a689af0f7f0ed352146f6769d8e85c5dfeb5fb8e51"}
Nov 02 02:48:20.548047 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.548024    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j" event={"ID":"e1ebd107-87fa-4417-98bc-68c9bcb531e6","Type":"ContainerStarted","Data":"92fad09ca86142d5a8439664580abb9fcc98614c5b173a4fdf0ddb825dbdcd17"}
Nov 02 02:48:20.548134 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.548069    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j" event={"ID":"e1ebd107-87fa-4417-98bc-68c9bcb531e6","Type":"ContainerStarted","Data":"071427d8830bd141e1eed9d6c0c36a295f8458793f8c44d890fb2c1fdaaa8488"}
Nov 02 02:48:20.548134 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.548086    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j" event={"ID":"e1ebd107-87fa-4417-98bc-68c9bcb531e6","Type":"ContainerStarted","Data":"487aea6e971df87f9965f959a8cd254113d8f78f0f7e819d2a7aeb60caaabfe2"}
Nov 02 02:48:20.552179 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552157    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 02 02:48:20.552271 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552166    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 02 02:48:20.552271 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552208    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 02 02:48:20.552271 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552261    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 02 02:48:20.552421 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552273    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 02 02:48:20.552421 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552314    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 02 02:48:20.552421 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552347    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 02 02:48:20.552421 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552361    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 02 02:48:20.552577 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.552544    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 02 02:48:20.602666 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.602599    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 02 02:48:20.631049 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631019    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631182 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631058    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckc5x\" (UniqueName: \"kubernetes.io/projected/b9f9767f-d787-4e76-bea2-26669d01a593-kube-api-access-ckc5x\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631182 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631100    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9f9767f-d787-4e76-bea2-26669d01a593-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631182 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631131    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631182 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631169    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631395 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631223    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-config\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631395 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631247    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9f9767f-d787-4e76-bea2-26669d01a593-config-out\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631395 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631273    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631395 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631299    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631557 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631542    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631746 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631725    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631817 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631783    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631817 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631810    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-web-config\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631920 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631860    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.631973 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.631960    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.632057 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.632042    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.632114 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.632093    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.632550 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.632117    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.632550 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.632180    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.632550 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.632263    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.634378 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.634347    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.634613 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.634588    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.635099 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.635002    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.635392 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.635366    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.636969 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.636823    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9f9767f-d787-4e76-bea2-26669d01a593-config-out\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.636969 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.636839    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.637499 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.637469    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.638185 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.638111    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.638657 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.638610    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.639021 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.638928    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.639143 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.639038    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.639401 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.639377    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-web-config\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.639821 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.639801    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.640430 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.640402    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b9f9767f-d787-4e76-bea2-26669d01a593-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.642547 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.642523    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.642828 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.642805    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.643196 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.643176    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-config\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.643748 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.643722    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9f9767f-d787-4e76-bea2-26669d01a593-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.643873 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.643857    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b9f9767f-d787-4e76-bea2-26669d01a593-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.688061 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.688041    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckc5x\" (UniqueName: \"kubernetes.io/projected/b9f9767f-d787-4e76-bea2-26669d01a593-kube-api-access-ckc5x\") pod \"prometheus-k8s-0\" (UID: \"b9f9767f-d787-4e76-bea2-26669d01a593\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.710907 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.710883    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:20.739313 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.739258    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5bddbb5fbf-rdj2j" podStartSLOduration=2.682202334 podStartE2EDuration="4.739230109s" totalImagesPullingTime="2.057027775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:48:20.737605921 +0000 UTC m=+44.532954488" watchObservedRunningTime="2026-11-02 02:48:20.739230109 +0000 UTC m=+44.534578677"
Nov 02 02:48:20.860940 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:20.860849    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.525846087 podStartE2EDuration="7.860836123s" totalImagesPullingTime="4.322849265s" totalInitContainerRuntime="12.140771ms" isStatefulPod=false podCreationTimestamp="2026-11-02 02:48:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:48:20.859680603 +0000 UTC m=+44.655029167" watchObservedRunningTime="2026-11-02 02:48:20.860836123 +0000 UTC m=+44.656184749"
Nov 02 02:48:21.135818 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:21.135789    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 02:48:21.136913 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:48:21.136892    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f9767f_d787_4e76_bea2_26669d01a593.slice/crio-e4eb1bc47b6bab7ab01eb28b68bd594df7c11d20745de2b2ab73f7aacf8938be WatchSource:0}: Error finding container e4eb1bc47b6bab7ab01eb28b68bd594df7c11d20745de2b2ab73f7aacf8938be: Status 404 returned error can't find the container with id e4eb1bc47b6bab7ab01eb28b68bd594df7c11d20745de2b2ab73f7aacf8938be
Nov 02 02:48:21.553242 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:21.553207    4407 generic.go:356] "Generic (PLEG): container finished" podID="b9f9767f-d787-4e76-bea2-26669d01a593" containerID="67c1f3a65e1a524630f497825088b2c3163e5ae1eef6d3551fb7bab12a566d76" exitCode=0
Nov 02 02:48:21.553722 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:21.553294    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9f9767f-d787-4e76-bea2-26669d01a593","Type":"ContainerDied","Data":"67c1f3a65e1a524630f497825088b2c3163e5ae1eef6d3551fb7bab12a566d76"}
Nov 02 02:48:21.553722 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:21.553368    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9f9767f-d787-4e76-bea2-26669d01a593","Type":"ContainerStarted","Data":"e4eb1bc47b6bab7ab01eb28b68bd594df7c11d20745de2b2ab73f7aacf8938be"}
Nov 02 02:48:23.059131 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:23.059092    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:23.215293 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:23.215249    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:23.373794 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:23.373699    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:24.725990 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:24.725939    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:24.881771 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:24.881734    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:25.716949 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:25.716873    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:25.869026 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:25.868977    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:26.577019 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:26.576977    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9f9767f-d787-4e76-bea2-26669d01a593","Type":"ContainerStarted","Data":"82045928969dc3893369117538331126137e98c81894e11ee64501912f5b4569"}
Nov 02 02:48:26.577187 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:26.577057    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9f9767f-d787-4e76-bea2-26669d01a593","Type":"ContainerStarted","Data":"ffd116e4d82275d45f81890a7abf304804aa73b2bbff18e74384fb69cd73b749"}
Nov 02 02:48:27.204077 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:27.204043    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:27.355806 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:27.355759    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:28.225544 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:28.225497    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:28.411299 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:28.411269    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:29.590945 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:29.590840    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9f9767f-d787-4e76-bea2-26669d01a593","Type":"ContainerStarted","Data":"0ab2493f3b55fa57d05444f4d3e9e9bab271802c3ac54fc41bc6349e747cd069"}
Nov 02 02:48:29.590945 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:29.590912    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9f9767f-d787-4e76-bea2-26669d01a593","Type":"ContainerStarted","Data":"e1aea887d01a710042933fec2b6725fe4fdd8921092b69f962927078d1bde583"}
Nov 02 02:48:30.598392 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:30.598364    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9f9767f-d787-4e76-bea2-26669d01a593","Type":"ContainerStarted","Data":"40425783d527ad9949ff126be87e7a98ab7eda9645abfbf3a27fa167d36d75d9"}
Nov 02 02:48:30.598782 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:30.598423    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9f9767f-d787-4e76-bea2-26669d01a593","Type":"ContainerStarted","Data":"a36e57ec513cad24ba860f07b26edccd098cc0416a1f265d90908e8bcc29538b"}
Nov 02 02:48:30.703786 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:30.703758    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:30.711848 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:30.711824    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:30.830601 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:30.830550    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.094824329 podStartE2EDuration="10.830537778s" totalImagesPullingTime="7.720778562s" totalInitContainerRuntime="14.934887ms" isStatefulPod=false podCreationTimestamp="2026-11-02 02:48:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:48:30.779086701 +0000 UTC m=+54.574435266" watchObservedRunningTime="2026-11-02 02:48:30.830537778 +0000 UTC m=+54.625886361"
Nov 02 02:48:30.860447 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:30.860373    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:31.948040 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:31.948005    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:32.113863 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:32.113833    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:34.319722 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:34.319688    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:34.489488 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:34.489455    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:35.188490 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:48:35.188451    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:48:38.275908 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:38.275869    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:40.693188 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:40.693149    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:41.381251 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:41.381213    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:41.549241 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:41.549211    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:43.750435 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:43.750390    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f" podUID="9b8d445b-011f-4a47-987e-8d54a0a11cc7" containerName="registry" containerID="cri-o://37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba" gracePeriod=30
Nov 02 02:48:44.002821 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.002767    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:48:44.048342 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048306    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-kube-api-access-wxcl2\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-kube-api-access-wxcl2\") pod \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") "
Nov 02 02:48:44.048519 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048360    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-trusted-ca\") pod \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") "
Nov 02 02:48:44.048519 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048387    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-bound-sa-token\") pod \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") "
Nov 02 02:48:44.048519 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048416    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-tls\") pod \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") "
Nov 02 02:48:44.048519 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048441    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b8d445b-011f-4a47-987e-8d54a0a11cc7-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8d445b-011f-4a47-987e-8d54a0a11cc7-ca-trust-extracted\") pod \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") "
Nov 02 02:48:44.048519 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048476    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-certificates\") pod \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") "
Nov 02 02:48:44.048519 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048517    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-installation-pull-secrets\") pod \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") "
Nov 02 02:48:44.048895 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048599    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-image-registry-private-configuration\") pod \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\" (UID: \"9b8d445b-011f-4a47-987e-8d54a0a11cc7\") "
Nov 02 02:48:44.048895 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.048854    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-trusted-ca" pod "9b8d445b-011f-4a47-987e-8d54a0a11cc7" (UID: "9b8d445b-011f-4a47-987e-8d54a0a11cc7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:48:44.049260 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.049234    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-certificates" pod "9b8d445b-011f-4a47-987e-8d54a0a11cc7" (UID: "9b8d445b-011f-4a47-987e-8d54a0a11cc7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 02:48:44.051026 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.051002    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-kube-api-access-wxcl2" pod "9b8d445b-011f-4a47-987e-8d54a0a11cc7" (UID: "9b8d445b-011f-4a47-987e-8d54a0a11cc7"). InnerVolumeSpecName "kube-api-access-wxcl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:48:44.051159 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.051138    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-bound-sa-token" pod "9b8d445b-011f-4a47-987e-8d54a0a11cc7" (UID: "9b8d445b-011f-4a47-987e-8d54a0a11cc7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:48:44.051284 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.051266    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-image-registry-private-configuration" pod "9b8d445b-011f-4a47-987e-8d54a0a11cc7" (UID: "9b8d445b-011f-4a47-987e-8d54a0a11cc7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:48:44.051442 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.051421    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-tls" pod "9b8d445b-011f-4a47-987e-8d54a0a11cc7" (UID: "9b8d445b-011f-4a47-987e-8d54a0a11cc7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 02:48:44.051442 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.051432    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-installation-pull-secrets" pod "9b8d445b-011f-4a47-987e-8d54a0a11cc7" (UID: "9b8d445b-011f-4a47-987e-8d54a0a11cc7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 02:48:44.055771 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.055746    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b8d445b-011f-4a47-987e-8d54a0a11cc7-ca-trust-extracted" pod "9b8d445b-011f-4a47-987e-8d54a0a11cc7" (UID: "9b8d445b-011f-4a47-987e-8d54a0a11cc7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 02:48:44.149799 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.149767    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxcl2\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-kube-api-access-wxcl2\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:48:44.149799 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.149796    4407 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-trusted-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:48:44.149799 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.149806    4407 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-bound-sa-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:48:44.150042 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.149815    4407 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:48:44.150042 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.149824    4407 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8d445b-011f-4a47-987e-8d54a0a11cc7-ca-trust-extracted\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:48:44.150042 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.149832    4407 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8d445b-011f-4a47-987e-8d54a0a11cc7-registry-certificates\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:48:44.150042 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.149840    4407 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-installation-pull-secrets\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:48:44.150042 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.149849    4407 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8d445b-011f-4a47-987e-8d54a0a11cc7-image-registry-private-configuration\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 02:48:44.649883 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.649793    4407 generic.go:356] "Generic (PLEG): container finished" podID="9b8d445b-011f-4a47-987e-8d54a0a11cc7" containerID="37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba" exitCode=0
Nov 02 02:48:44.649883 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.649874    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f"
Nov 02 02:48:44.650105 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.649879    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f" event={"ID":"9b8d445b-011f-4a47-987e-8d54a0a11cc7","Type":"ContainerDied","Data":"37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba"}
Nov 02 02:48:44.650105 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.649944    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6654798c4f-2sp4f" event={"ID":"9b8d445b-011f-4a47-987e-8d54a0a11cc7","Type":"ContainerDied","Data":"659623cdd3f7a316d09e10c398c92f060fee5aa75ec2a089464bc486abe759fd"}
Nov 02 02:48:44.650105 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.649971    4407 scope.go:122] "RemoveContainer" containerID="37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba"
Nov 02 02:48:44.658061 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.658033    4407 scope.go:122] "RemoveContainer" containerID="37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba"
Nov 02 02:48:44.658316 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:48:44.658294    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba\": container with ID starting with 37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba not found: ID does not exist" containerID="37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba"
Nov 02 02:48:44.658366 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.658326    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba"} err="failed to get container status \"37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba\": rpc error: code = NotFound desc = could not find container \"37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba\": container with ID starting with 37b9e90c1cbf862f599bfad983bc9acd31130cefcd0999bad78d15df13ebeeba not found: ID does not exist"
Nov 02 02:48:44.814479 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.814440    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6654798c4f-2sp4f"]
Nov 02 02:48:44.865152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.865115    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6654798c4f-2sp4f"]
Nov 02 02:48:44.897349 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.897324    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:44.915911 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:44.915841    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:45.669824 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:45.669799    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 02:48:46.348771 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:46.348739    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b8d445b-011f-4a47-987e-8d54a0a11cc7" path="/var/lib/kubelet/pods/9b8d445b-011f-4a47-987e-8d54a0a11cc7/volumes"
Nov 02 02:48:52.497242 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:52.497208    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:52.660032 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:52.659999    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:58.029784 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:58.029745    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:48:58.186722 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:48:58.186692    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:01.447418 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:01.447381    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:01.604701 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:01.604668    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:02.593349 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:02.593301    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:02.749974 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:02.749942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:02.993493 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:02.993449    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:03.146648 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:03.146595    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:05.187372 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:49:05.187339    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:49:15.717261 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:15.717228    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:15.868876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:15.868838    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:28.630240 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:49:28.630200    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:49:35.188537 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:49:35.188499    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:50:05.187607 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:50:05.187561    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:50:18.010362 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:50:18.010325    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:50:28.919069 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:50:28.919035    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:50:35.187709 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:50:35.187672    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:51:05.187864 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:51:05.187818    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:51:17.175235 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:17.175199    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:17.494470 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:17.494448    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:18.471259 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:18.471227    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:18.631421 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:18.631385    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:19.507370 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:19.507337    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:24.641919 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:24.641863    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:24.814676 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:24.814641    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:25.661577 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:25.661541    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:25.972680 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:25.972572    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:26.128842 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:26.128815    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:26.640448 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:26.640415    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:26.799693 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:26.799660    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:29.253811 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:29.253775    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:29.489509 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:29.489455    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:29.670290 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:29.670212    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:35.187605 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:51:35.187565    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:51:57.995470 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:57.995407    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:59.116768 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.116734    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"]
Nov 02 02:51:59.117149 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.117034    4407 cpu_manager.go:414] "containerMap: removing container" podUID="9b8d445b-011f-4a47-987e-8d54a0a11cc7" containerName="registry"
Nov 02 02:51:59.117149 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.117044    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b8d445b-011f-4a47-987e-8d54a0a11cc7" containerName="registry"
Nov 02 02:51:59.117149 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.117091    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b8d445b-011f-4a47-987e-8d54a0a11cc7" containerName="registry"
Nov 02 02:51:59.119963 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.119947    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.165842 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.165813    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.165962 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.165882    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20912ef9-7741-4828-918a-2f177eba3c17-manifests\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.165962 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.165924    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjcl\" (UniqueName: \"kubernetes.io/projected/20912ef9-7741-4828-918a-2f177eba3c17-kube-api-access-hdjcl\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.166036 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.166017    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.166069 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.166041    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.166069 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.166062    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20912ef9-7741-4828-918a-2f177eba3c17-tmp\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.210790 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.210753    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 02 02:51:59.210790 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.210775    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 02 02:51:59.223995 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.223940    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 02 02:51:59.224126 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.224056    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-2r6bp\""
Nov 02 02:51:59.224126 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.224117    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 02 02:51:59.224247 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.224124    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 02 02:51:59.266866 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.266837    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.267045 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.266957    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20912ef9-7741-4828-918a-2f177eba3c17-manifests\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.267045 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.266987    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjcl\" (UniqueName: \"kubernetes.io/projected/20912ef9-7741-4828-918a-2f177eba3c17-kube-api-access-hdjcl\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.267045 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.267017    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.267325 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.267050    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.267325 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.267080    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20912ef9-7741-4828-918a-2f177eba3c17-tmp\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.267434 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.267361    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20912ef9-7741-4828-918a-2f177eba3c17-manifests\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.269466 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.269437    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20912ef9-7741-4828-918a-2f177eba3c17-tmp\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.269872 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.269851    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.269947 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.269851    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.269947 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.269858    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20912ef9-7741-4828-918a-2f177eba3c17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.327209 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.327177    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjcl\" (UniqueName: \"kubernetes.io/projected/20912ef9-7741-4828-918a-2f177eba3c17-kube-api-access-hdjcl\") pod \"opendatahub-operator-controller-manager-5b55749cb7-zt7rl\" (UID: \"20912ef9-7741-4828-918a-2f177eba3c17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.364418 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.364387    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"]
Nov 02 02:51:59.434534 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.434452    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:51:59.700262 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.700194    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:51:59.916036 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:51:59.916012    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"]
Nov 02 02:51:59.917936 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:51:59.917909    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20912ef9_7741_4828_918a_2f177eba3c17.slice/crio-dc67bce0b3efab400d49af1f4a5eb42c4b6984c25ef8e62ebccf18360d51c937 WatchSource:0}: Error finding container dc67bce0b3efab400d49af1f4a5eb42c4b6984c25ef8e62ebccf18360d51c937: Status 404 returned error can't find the container with id dc67bce0b3efab400d49af1f4a5eb42c4b6984c25ef8e62ebccf18360d51c937
Nov 02 02:52:00.305293 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:00.305263    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl" event={"ID":"20912ef9-7741-4828-918a-2f177eba3c17","Type":"ContainerStarted","Data":"dc67bce0b3efab400d49af1f4a5eb42c4b6984c25ef8e62ebccf18360d51c937"}
Nov 02 02:52:01.593986 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:01.593951    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:01.746871 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:01.746830    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:03.164649 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:03.164585    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:03.345870 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:03.345840    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:04.325788 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:04.325753    4407 generic.go:356] "Generic (PLEG): container finished" podID="20912ef9-7741-4828-918a-2f177eba3c17" containerID="cd3b90de14715c5b15883a6c573a2192d692ba1c07aacd04c858725d609dcb78" exitCode=0
Nov 02 02:52:04.326158 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:04.325802    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl" event={"ID":"20912ef9-7741-4828-918a-2f177eba3c17","Type":"ContainerDied","Data":"cd3b90de14715c5b15883a6c573a2192d692ba1c07aacd04c858725d609dcb78"}
Nov 02 02:52:04.977638 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:04.977591    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:05.191583 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:52:05.191550    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:52:05.330858 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:05.330823    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl" event={"ID":"20912ef9-7741-4828-918a-2f177eba3c17","Type":"ContainerStarted","Data":"d94a94d214d9bee5d16204e8c517b1788ae99971de6902a87fee1d02883d3c19"}
Nov 02 02:52:05.331268 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:05.330948    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:52:05.568747 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:05.568696    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl" podStartSLOduration=2.834861773 podStartE2EDuration="7.568680401s" totalImagesPullingTime="4.316442247s" totalInitContainerRuntime="417.376381ms" isStatefulPod=false podCreationTimestamp="2026-11-02 02:51:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:52:05.567818827 +0000 UTC m=+269.363167393" watchObservedRunningTime="2026-11-02 02:52:05.568680401 +0000 UTC m=+269.364028966"
Nov 02 02:52:05.587987 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:05.587928    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:05.748645 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:05.748597    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:09.575538 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:09.575504    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:10.383426 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:10.383394    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:11.771782 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:11.771750    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:14.817046 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:14.817010    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:15.002924 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:15.002888    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:15.181403 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:15.181310    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:16.336923 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:16.336887    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5b55749cb7-zt7rl"
Nov 02 02:52:16.850120 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:16.850090    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:17.407785 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:17.407736    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:18.167904 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.167872    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"]
Nov 02 02:52:18.170981 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.170964    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"
Nov 02 02:52:18.222927 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.222891    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lbg7\" (UniqueName: \"kubernetes.io/projected/6f475a0e-297e-4609-a97c-8af20c647fb2-kube-api-access-9lbg7\") pod \"limitador-operator-controller-manager-5778bb4449-dsph6\" (UID: \"6f475a0e-297e-4609-a97c-8af20c647fb2\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"
Nov 02 02:52:18.225203 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.225184    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-976d2\""
Nov 02 02:52:18.275994 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.275954    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 02 02:52:18.275994 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.275963    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 02 02:52:18.277207 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.277188    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"]
Nov 02 02:52:18.323951 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.323919    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lbg7\" (UniqueName: \"kubernetes.io/projected/6f475a0e-297e-4609-a97c-8af20c647fb2-kube-api-access-9lbg7\") pod \"limitador-operator-controller-manager-5778bb4449-dsph6\" (UID: \"6f475a0e-297e-4609-a97c-8af20c647fb2\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"
Nov 02 02:52:18.384876 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.384845    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lbg7\" (UniqueName: \"kubernetes.io/projected/6f475a0e-297e-4609-a97c-8af20c647fb2-kube-api-access-9lbg7\") pod \"limitador-operator-controller-manager-5778bb4449-dsph6\" (UID: \"6f475a0e-297e-4609-a97c-8af20c647fb2\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"
Nov 02 02:52:18.482637 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.482535    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"
Nov 02 02:52:18.852827 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.852766    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"]
Nov 02 02:52:18.855068 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:52:18.855041    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f475a0e_297e_4609_a97c_8af20c647fb2.slice/crio-ad2242747043befe0570800f74d99272d140c4746adfb81b467691ee2e97e092 WatchSource:0}: Error finding container ad2242747043befe0570800f74d99272d140c4746adfb81b467691ee2e97e092: Status 404 returned error can't find the container with id ad2242747043befe0570800f74d99272d140c4746adfb81b467691ee2e97e092
Nov 02 02:52:18.968901 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:18.968870    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:19.382348 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:19.382316    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6" event={"ID":"6f475a0e-297e-4609-a97c-8af20c647fb2","Type":"ContainerStarted","Data":"ad2242747043befe0570800f74d99272d140c4746adfb81b467691ee2e97e092"}
Nov 02 02:52:19.803013 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:19.802974    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:20.008089 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:20.008038    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:20.577839 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:20.577807    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:21.862450 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:21.862415    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:22.174354 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.174277    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:22.424715 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.424596    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:22.659452 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.659387    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:22.823637 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.823587    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"]
Nov 02 02:52:22.834417 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.834393    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:22.877983 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.877946    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:22.888110 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.888084    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"]
Nov 02 02:52:22.937931 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.937898    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 02 02:52:22.938081 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.937955    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 02 02:52:22.964093 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.964061    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4320f43d-b796-4644-8b17-9f3b9c1bbfd6-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-wqbbg\" (UID: \"4320f43d-b796-4644-8b17-9f3b9c1bbfd6\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:22.964258 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.964191    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4vmp\" (UniqueName: \"kubernetes.io/projected/4320f43d-b796-4644-8b17-9f3b9c1bbfd6-kube-api-access-k4vmp\") pod \"cert-manager-webhook-5dbbbf75cc-wqbbg\" (UID: \"4320f43d-b796-4644-8b17-9f3b9c1bbfd6\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:22.988750 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:22.988718    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hg66j\""
Nov 02 02:52:23.065344 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.065310    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4320f43d-b796-4644-8b17-9f3b9c1bbfd6-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-wqbbg\" (UID: \"4320f43d-b796-4644-8b17-9f3b9c1bbfd6\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:23.065527 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.065395    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4vmp\" (UniqueName: \"kubernetes.io/projected/4320f43d-b796-4644-8b17-9f3b9c1bbfd6-kube-api-access-k4vmp\") pod \"cert-manager-webhook-5dbbbf75cc-wqbbg\" (UID: \"4320f43d-b796-4644-8b17-9f3b9c1bbfd6\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:23.132666 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.132565    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4320f43d-b796-4644-8b17-9f3b9c1bbfd6-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-wqbbg\" (UID: \"4320f43d-b796-4644-8b17-9f3b9c1bbfd6\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:23.132666 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.132614    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4vmp\" (UniqueName: \"kubernetes.io/projected/4320f43d-b796-4644-8b17-9f3b9c1bbfd6-kube-api-access-k4vmp\") pod \"cert-manager-webhook-5dbbbf75cc-wqbbg\" (UID: \"4320f43d-b796-4644-8b17-9f3b9c1bbfd6\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:23.165700 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.165443    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:23.398511 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.398432    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6" event={"ID":"6f475a0e-297e-4609-a97c-8af20c647fb2","Type":"ContainerStarted","Data":"05a2f8930ad70706487e80821fa22217242df3e441676911747baa97ba0811c0"}
Nov 02 02:52:23.398682 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.398553    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"
Nov 02 02:52:23.538535 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.538481    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6" podStartSLOduration=1.110314572 podStartE2EDuration="5.538468646s" totalImagesPullingTime="4.428154074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:52:23.5367915 +0000 UTC m=+287.332140065" watchObservedRunningTime="2026-11-02 02:52:23.538468646 +0000 UTC m=+287.333817210"
Nov 02 02:52:23.569081 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:23.569063    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"]
Nov 02 02:52:23.570708 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:52:23.570680    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4320f43d_b796_4644_8b17_9f3b9c1bbfd6.slice/crio-7b1ef060d7e68748f7f2f0fdb2c410abef1d156ad5d3f576ea7c73feb5596123 WatchSource:0}: Error finding container 7b1ef060d7e68748f7f2f0fdb2c410abef1d156ad5d3f576ea7c73feb5596123: Status 404 returned error can't find the container with id 7b1ef060d7e68748f7f2f0fdb2c410abef1d156ad5d3f576ea7c73feb5596123
Nov 02 02:52:24.403404 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:24.403364    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg" event={"ID":"4320f43d-b796-4644-8b17-9f3b9c1bbfd6","Type":"ContainerStarted","Data":"7b1ef060d7e68748f7f2f0fdb2c410abef1d156ad5d3f576ea7c73feb5596123"}
Nov 02 02:52:24.937400 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:24.937358    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:25.121437 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.121407    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"]
Nov 02 02:52:25.124793 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.124777    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"
Nov 02 02:52:25.183763 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.183734    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"]
Nov 02 02:52:25.183928 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.183905    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ee2bae5-9219-4e80-9d97-a674cfafd4e8-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sdz2h\" (UID: \"3ee2bae5-9219-4e80-9d97-a674cfafd4e8\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"
Nov 02 02:52:25.184045 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.184031    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wl9v\" (UniqueName: \"kubernetes.io/projected/3ee2bae5-9219-4e80-9d97-a674cfafd4e8-kube-api-access-9wl9v\") pod \"cert-manager-cainjector-5c57bbd4fc-sdz2h\" (UID: \"3ee2bae5-9219-4e80-9d97-a674cfafd4e8\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"
Nov 02 02:52:25.232206 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.232141    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xdfrj\""
Nov 02 02:52:25.284425 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.284396    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ee2bae5-9219-4e80-9d97-a674cfafd4e8-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sdz2h\" (UID: \"3ee2bae5-9219-4e80-9d97-a674cfafd4e8\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"
Nov 02 02:52:25.284570 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.284456    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wl9v\" (UniqueName: \"kubernetes.io/projected/3ee2bae5-9219-4e80-9d97-a674cfafd4e8-kube-api-access-9wl9v\") pod \"cert-manager-cainjector-5c57bbd4fc-sdz2h\" (UID: \"3ee2bae5-9219-4e80-9d97-a674cfafd4e8\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"
Nov 02 02:52:25.301513 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.301489    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:25.343733 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.343699    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ee2bae5-9219-4e80-9d97-a674cfafd4e8-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sdz2h\" (UID: \"3ee2bae5-9219-4e80-9d97-a674cfafd4e8\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"
Nov 02 02:52:25.344183 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.344167    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wl9v\" (UniqueName: \"kubernetes.io/projected/3ee2bae5-9219-4e80-9d97-a674cfafd4e8-kube-api-access-9wl9v\") pod \"cert-manager-cainjector-5c57bbd4fc-sdz2h\" (UID: \"3ee2bae5-9219-4e80-9d97-a674cfafd4e8\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"
Nov 02 02:52:25.436440 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.436411    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"
Nov 02 02:52:25.815430 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:25.815403    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h"]
Nov 02 02:52:25.817989 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:52:25.817958    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ee2bae5_9219_4e80_9d97_a674cfafd4e8.slice/crio-6a882fc98e5a65df097557389c3237901287bd0acccafdaaff46cef9a29cec58 WatchSource:0}: Error finding container 6a882fc98e5a65df097557389c3237901287bd0acccafdaaff46cef9a29cec58: Status 404 returned error can't find the container with id 6a882fc98e5a65df097557389c3237901287bd0acccafdaaff46cef9a29cec58
Nov 02 02:52:26.411822 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:26.411786    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h" event={"ID":"3ee2bae5-9219-4e80-9d97-a674cfafd4e8","Type":"ContainerStarted","Data":"6a882fc98e5a65df097557389c3237901287bd0acccafdaaff46cef9a29cec58"}
Nov 02 02:52:28.199614 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:28.199581    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:28.367444 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:28.367419    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:29.013285 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:29.013244    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:29.244600 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:29.244523    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:29.603179 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:29.603049    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:29.773216 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:29.773120    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:31.277614 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.277512    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:31.281152 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.281124    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"]
Nov 02 02:52:31.291819 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.291786    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.344288 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.341837    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"]
Nov 02 02:52:31.385761 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.385157    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 02:52:31.390512 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.390472    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 02:52:31.391286 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.391258    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 02 02:52:31.391433 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.391417    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cksvz\""
Nov 02 02:52:31.395557 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.395537    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 02 02:52:31.437218 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.437184    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-tls-certs\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.437218 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.437229    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-tmp\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.437459 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.437428    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgn6f\" (UniqueName: \"kubernetes.io/projected/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-kube-api-access-hgn6f\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.488217 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.488178    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:31.538033 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.537992    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgn6f\" (UniqueName: \"kubernetes.io/projected/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-kube-api-access-hgn6f\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.538203 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.538042    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-tls-certs\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.538203 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.538071    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-tmp\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.540970 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.540943    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-tmp\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.541086 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.541053    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-tls-certs\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.596124 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.596064    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgn6f\" (UniqueName: \"kubernetes.io/projected/f4c44847-af85-4769-9ca1-1b03e7fbdf4e-kube-api-access-hgn6f\") pod \"kube-auth-proxy-66599bc4bb-sx4rr\" (UID: \"f4c44847-af85-4769-9ca1-1b03e7fbdf4e\") " pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.608860 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.608829    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"
Nov 02 02:52:31.650215 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.650138    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:31.942004 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:31.941923    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:32.001159 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:32.001135    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr"]
Nov 02 02:52:32.004038 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:52:32.004008    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c44847_af85_4769_9ca1_1b03e7fbdf4e.slice/crio-70e3db4288ccbf96810c229d137f366281f99f7c03157c21108fc9d0044aef9b WatchSource:0}: Error finding container 70e3db4288ccbf96810c229d137f366281f99f7c03157c21108fc9d0044aef9b: Status 404 returned error can't find the container with id 70e3db4288ccbf96810c229d137f366281f99f7c03157c21108fc9d0044aef9b
Nov 02 02:52:32.202868 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:32.202843    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:32.442552 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:32.442512    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg" event={"ID":"4320f43d-b796-4644-8b17-9f3b9c1bbfd6","Type":"ContainerStarted","Data":"1c25a6f3b2798571fbb744d8e0f42e4091658fed1765f73753d88a5a5b52ce3e"}
Nov 02 02:52:32.443005 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:32.442601    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:32.444159 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:32.444138    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h" event={"ID":"3ee2bae5-9219-4e80-9d97-a674cfafd4e8","Type":"ContainerStarted","Data":"ddab055c7fdfedc8cf1b39fbd4beadd20c28afbc1e3790f257701be995704290"}
Nov 02 02:52:32.445301 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:32.445283    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr" event={"ID":"f4c44847-af85-4769-9ca1-1b03e7fbdf4e","Type":"ContainerStarted","Data":"70e3db4288ccbf96810c229d137f366281f99f7c03157c21108fc9d0044aef9b"}
Nov 02 02:52:32.559531 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:32.559481    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg" podStartSLOduration=1.967051501 podStartE2EDuration="10.559467925s" totalImagesPullingTime="8.592416424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:52:32.558504281 +0000 UTC m=+296.353852847" watchObservedRunningTime="2026-11-02 02:52:32.559467925 +0000 UTC m=+296.354816489"
Nov 02 02:52:32.680357 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:32.680305    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdz2h" podStartSLOduration=1.1973393159999999 podStartE2EDuration="7.68029201s" totalImagesPullingTime="6.482952694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:52:32.680070114 +0000 UTC m=+296.475418679" watchObservedRunningTime="2026-11-02 02:52:32.68029201 +0000 UTC m=+296.475640575"
Nov 02 02:52:33.150690 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:33.150605    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:33.975343 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:33.975310    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:34.136267 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:34.136233    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:34.406066 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:34.406029    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dsph6"
Nov 02 02:52:34.583932 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:34.583904    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:34.761661 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:34.761613    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:35.187800 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:52:35.187728    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:52:35.465873 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:35.465782    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr" event={"ID":"f4c44847-af85-4769-9ca1-1b03e7fbdf4e","Type":"ContainerStarted","Data":"63055812f14772fca3c815b586acf7ff629327bfde8cc0152369de3a1e9c2601"}
Nov 02 02:52:35.582345 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:35.582296    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66599bc4bb-sx4rr" podStartSLOduration=1.647986499 podStartE2EDuration="4.582283882s" totalImagesPullingTime="2.934297383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:52:35.58219412 +0000 UTC m=+299.377542685" watchObservedRunningTime="2026-11-02 02:52:35.582283882 +0000 UTC m=+299.377632447"
Nov 02 02:52:38.439775 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:38.439718    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:38.453354 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:38.453327    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-wqbbg"
Nov 02 02:52:38.629849 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:38.629819    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:38.910009 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:38.909927    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:43.115035 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:43.115003    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:43.268723 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:43.268687    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:43.915922 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:43.915890    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"]
Nov 02 02:52:43.920694 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:43.920676    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:43.974941 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:43.974911    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 02 02:52:44.025385 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.025345    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 02 02:52:44.025565 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.025489    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-9xghq\""
Nov 02 02:52:44.047868 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.047834    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0498751-0dc7-4bc2-9076-a9c0c9411a46-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.048026 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.047876    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0498751-0dc7-4bc2-9076-a9c0c9411a46-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.048026 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.047906    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tht54\" (UniqueName: \"kubernetes.io/projected/a0498751-0dc7-4bc2-9076-a9c0c9411a46-kube-api-access-tht54\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.048026 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.047994    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0498751-0dc7-4bc2-9076-a9c0c9411a46-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.127478 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.127447    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 02 02:52:44.128938 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.128916    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"]
Nov 02 02:52:44.148998 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.148975    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0498751-0dc7-4bc2-9076-a9c0c9411a46-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.149079 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.149006    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tht54\" (UniqueName: \"kubernetes.io/projected/a0498751-0dc7-4bc2-9076-a9c0c9411a46-kube-api-access-tht54\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.149079 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.149035    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0498751-0dc7-4bc2-9076-a9c0c9411a46-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.149215 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.149080    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0498751-0dc7-4bc2-9076-a9c0c9411a46-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.149433 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.149412    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0498751-0dc7-4bc2-9076-a9c0c9411a46-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.149471 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.149433    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0498751-0dc7-4bc2-9076-a9c0c9411a46-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.149870 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.149853    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0498751-0dc7-4bc2-9076-a9c0c9411a46-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.207512 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.207445    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tht54\" (UniqueName: \"kubernetes.io/projected/a0498751-0dc7-4bc2-9076-a9c0c9411a46-kube-api-access-tht54\") pod \"kserve-module-controller-manager-7566cf5fc8-2f5bd\" (UID: \"a0498751-0dc7-4bc2-9076-a9c0c9411a46\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.233345 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.233317    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:44.654299 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.654276    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"]
Nov 02 02:52:44.656749 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:52:44.656723    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0498751_0dc7_4bc2_9076_a9c0c9411a46.slice/crio-6f1f3cd60ba26e41374a81e00d8e05fe2e306bc32b0616a31c6fd77e6a771354 WatchSource:0}: Error finding container 6f1f3cd60ba26e41374a81e00d8e05fe2e306bc32b0616a31c6fd77e6a771354: Status 404 returned error can't find the container with id 6f1f3cd60ba26e41374a81e00d8e05fe2e306bc32b0616a31c6fd77e6a771354
Nov 02 02:52:44.658709 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:44.658690    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 02:52:45.505462 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:45.505427    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd" event={"ID":"a0498751-0dc7-4bc2-9076-a9c0c9411a46","Type":"ContainerStarted","Data":"6f1f3cd60ba26e41374a81e00d8e05fe2e306bc32b0616a31c6fd77e6a771354"}
Nov 02 02:52:47.514101 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:47.514063    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd" event={"ID":"a0498751-0dc7-4bc2-9076-a9c0c9411a46","Type":"ContainerStarted","Data":"09014cea710178ea840bd30673b9291a86cabc822309e318e96a3d1f095d0b29"}
Nov 02 02:52:47.514487 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:47.514201    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:52:47.682019 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:47.681964    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd" podStartSLOduration=2.693070162 podStartE2EDuration="4.681949801s" totalImagesPullingTime="1.988879639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:52:47.681219124 +0000 UTC m=+311.476567691" watchObservedRunningTime="2026-11-02 02:52:47.681949801 +0000 UTC m=+311.477298394"
Nov 02 02:52:49.803819 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:49.803790    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:49.958582 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:49.958550    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:55.195900 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:55.195869    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:55.347216 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:55.347184    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:55.596709 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:55.596677    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:55.752595 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:55.752565    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:58.996324 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:58.996290    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:52:59.159333 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:52:59.159296    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:02.791071 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:02.791035    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:02.942723 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:02.942690    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:05.083844 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:05.083809    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:05.187977 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:53:05.187937    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:53:05.242923 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:05.242895    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:05.400244 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:05.400169    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:08.519614 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:08.519585    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-2f5bd"
Nov 02 02:53:29.954395 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:29.954356    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:35.188797 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:53:35.188759    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:53:48.068979 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.068935    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:48.256680 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.256652    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:48.455995 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.455920    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:48.617910 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.617874    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:48.776832 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.776800    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:48.939419 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.939378    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-kh22l"]
Nov 02 02:53:48.942567 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.942545    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:48.951822 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.951799    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:48.998534 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:48.998503    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-kh22l"]
Nov 02 02:53:49.001431 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.001403    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kgn6n\""
Nov 02 02:53:49.005992 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.005958    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qbrc\" (UniqueName: \"kubernetes.io/projected/e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2-kube-api-access-4qbrc\") pod \"kserve-controller-manager-776cc66d47-kh22l\" (UID: \"e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:49.006101 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.006005    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2-cert\") pod \"kserve-controller-manager-776cc66d47-kh22l\" (UID: \"e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:49.047190 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.047124    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 02 02:53:49.106541 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.106510    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qbrc\" (UniqueName: \"kubernetes.io/projected/e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2-kube-api-access-4qbrc\") pod \"kserve-controller-manager-776cc66d47-kh22l\" (UID: \"e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:49.106997 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.106566    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2-cert\") pod \"kserve-controller-manager-776cc66d47-kh22l\" (UID: \"e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:49.109361 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.109337    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2-cert\") pod \"kserve-controller-manager-776cc66d47-kh22l\" (UID: \"e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:49.112230 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.112204    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:49.158378 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.158346    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-rhpmb"]
Nov 02 02:53:49.161780 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.161765    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:49.170134 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.170114    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qbrc\" (UniqueName: \"kubernetes.io/projected/e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2-kube-api-access-4qbrc\") pod \"kserve-controller-manager-776cc66d47-kh22l\" (UID: \"e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:49.207212 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.207179    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f48tg\" (UniqueName: \"kubernetes.io/projected/e4694bf5-7e68-4fff-8908-53345db86ffc-kube-api-access-f48tg\") pod \"model-serving-api-75bd89fdfc-rhpmb\" (UID: \"e4694bf5-7e68-4fff-8908-53345db86ffc\") " pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:49.207212 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.207216    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4694bf5-7e68-4fff-8908-53345db86ffc-tls-certs\") pod \"model-serving-api-75bd89fdfc-rhpmb\" (UID: \"e4694bf5-7e68-4fff-8908-53345db86ffc\") " pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:49.217228 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.217202    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 02 02:53:49.217409 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.217298    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-7zhq5\""
Nov 02 02:53:49.218662 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.218618    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-rhpmb"]
Nov 02 02:53:49.255218 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.255190    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:49.271234 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.271210    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-qj42d"]
Nov 02 02:53:49.274167 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.274143    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:49.275089 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.275070    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.312176 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.310044    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f48tg\" (UniqueName: \"kubernetes.io/projected/e4694bf5-7e68-4fff-8908-53345db86ffc-kube-api-access-f48tg\") pod \"model-serving-api-75bd89fdfc-rhpmb\" (UID: \"e4694bf5-7e68-4fff-8908-53345db86ffc\") " pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:49.312176 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.310096    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4694bf5-7e68-4fff-8908-53345db86ffc-tls-certs\") pod \"model-serving-api-75bd89fdfc-rhpmb\" (UID: \"e4694bf5-7e68-4fff-8908-53345db86ffc\") " pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:49.312176 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.310127    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhbfq\" (UniqueName: \"kubernetes.io/projected/feaad75e-40cc-46a5-83f7-586c25f24ab4-kube-api-access-lhbfq\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.312176 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.310175    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feaad75e-40cc-46a5-83f7-586c25f24ab4-metrics-certs\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.312176 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.310226    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/feaad75e-40cc-46a5-83f7-586c25f24ab4-cert\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.316802 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.313738    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4694bf5-7e68-4fff-8908-53345db86ffc-tls-certs\") pod \"model-serving-api-75bd89fdfc-rhpmb\" (UID: \"e4694bf5-7e68-4fff-8908-53345db86ffc\") " pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:49.328937 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.328758    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 02 02:53:49.328937 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.328870    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5zmwx\""
Nov 02 02:53:49.328937 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.328911    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 02 02:53:49.330299 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.330273    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-qj42d"]
Nov 02 02:53:49.371278 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.371239    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f48tg\" (UniqueName: \"kubernetes.io/projected/e4694bf5-7e68-4fff-8908-53345db86ffc-kube-api-access-f48tg\") pod \"model-serving-api-75bd89fdfc-rhpmb\" (UID: \"e4694bf5-7e68-4fff-8908-53345db86ffc\") " pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:49.410937 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.410906    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/feaad75e-40cc-46a5-83f7-586c25f24ab4-cert\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.411060 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.410967    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhbfq\" (UniqueName: \"kubernetes.io/projected/feaad75e-40cc-46a5-83f7-586c25f24ab4-kube-api-access-lhbfq\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.411060 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.411001    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feaad75e-40cc-46a5-83f7-586c25f24ab4-metrics-certs\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.413344 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.413322    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/feaad75e-40cc-46a5-83f7-586c25f24ab4-cert\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.413422 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.413331    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feaad75e-40cc-46a5-83f7-586c25f24ab4-metrics-certs\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.468696 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.468670    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhbfq\" (UniqueName: \"kubernetes.io/projected/feaad75e-40cc-46a5-83f7-586c25f24ab4-kube-api-access-lhbfq\") pod \"odh-model-controller-769875db56-qj42d\" (UID: \"feaad75e-40cc-46a5-83f7-586c25f24ab4\") " pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.474511 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.474476    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:49.597007 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.596600    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:49.627320 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.627275    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-kh22l"]
Nov 02 02:53:49.629164 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:53:49.629136    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39eb805_c6c1_4c36_8f29_9e9c0b7ac0f2.slice/crio-73604557485d792482ce254d27f5622200845948760d55d1d8c1704a0930330a WatchSource:0}: Error finding container 73604557485d792482ce254d27f5622200845948760d55d1d8c1704a0930330a: Status 404 returned error can't find the container with id 73604557485d792482ce254d27f5622200845948760d55d1d8c1704a0930330a
Nov 02 02:53:49.763498 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.763466    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l" event={"ID":"e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2","Type":"ContainerStarted","Data":"73604557485d792482ce254d27f5622200845948760d55d1d8c1704a0930330a"}
Nov 02 02:53:49.840610 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.840583    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-rhpmb"]
Nov 02 02:53:49.842537 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:53:49.842501    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4694bf5_7e68_4fff_8908_53345db86ffc.slice/crio-776f55e8f921a4502adc197642e0725b85a8394c1f6ba21a5d3605f1999ec816 WatchSource:0}: Error finding container 776f55e8f921a4502adc197642e0725b85a8394c1f6ba21a5d3605f1999ec816: Status 404 returned error can't find the container with id 776f55e8f921a4502adc197642e0725b85a8394c1f6ba21a5d3605f1999ec816
Nov 02 02:53:49.977195 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:49.977171    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-qj42d"]
Nov 02 02:53:49.978810 ip-10-0-131-112 kubenswrapper[4407]: W1102 02:53:49.978778    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaad75e_40cc_46a5_83f7_586c25f24ab4.slice/crio-775fb6644a7af6edc1aa83318be6cce1adef3957f819f3733901ec912f70b262 WatchSource:0}: Error finding container 775fb6644a7af6edc1aa83318be6cce1adef3957f819f3733901ec912f70b262: Status 404 returned error can't find the container with id 775fb6644a7af6edc1aa83318be6cce1adef3957f819f3733901ec912f70b262
Nov 02 02:53:50.329560 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:50.329524    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:50.770781 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:50.770745    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb" event={"ID":"e4694bf5-7e68-4fff-8908-53345db86ffc","Type":"ContainerStarted","Data":"776f55e8f921a4502adc197642e0725b85a8394c1f6ba21a5d3605f1999ec816"}
Nov 02 02:53:50.771920 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:50.771894    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-qj42d" event={"ID":"feaad75e-40cc-46a5-83f7-586c25f24ab4","Type":"ContainerStarted","Data":"775fb6644a7af6edc1aa83318be6cce1adef3957f819f3733901ec912f70b262"}
Nov 02 02:53:53.788591 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:53.788553    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb" event={"ID":"e4694bf5-7e68-4fff-8908-53345db86ffc","Type":"ContainerStarted","Data":"336fc721412b1ab183916b6d8018b4417b9dadca49c764e509eb5d230e89e47e"}
Nov 02 02:53:53.789094 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:53.788741    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:53:53.790289 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:53.790261    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l" event={"ID":"e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2","Type":"ContainerStarted","Data":"af109c65590a90607d7a04623592f058a52587f7793ab0c14ccabe81e1c3896f"}
Nov 02 02:53:53.790424 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:53.790357    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:53:53.791748 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:53.791724    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-qj42d" event={"ID":"feaad75e-40cc-46a5-83f7-586c25f24ab4","Type":"ContainerStarted","Data":"791594873bd1b48e8f5b9ec575c6df51e415d2c680e959283cf7dfb6545cdbd2"}
Nov 02 02:53:53.791865 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:53.791851    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:53:53.904645 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:53.904574    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb" podStartSLOduration=1.968092474 podStartE2EDuration="4.904562128s" totalImagesPullingTime="2.936469654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:53:53.903843647 +0000 UTC m=+377.699192210" watchObservedRunningTime="2026-11-02 02:53:53.904562128 +0000 UTC m=+377.699910690"
Nov 02 02:53:54.019675 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:54.019591    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l" podStartSLOduration=2.793126557 podStartE2EDuration="6.019577016s" totalImagesPullingTime="3.226450459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:53:54.018845691 +0000 UTC m=+377.814194256" watchObservedRunningTime="2026-11-02 02:53:54.019577016 +0000 UTC m=+377.814925578"
Nov 02 02:53:54.135299 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:54.135245    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-qj42d" podStartSLOduration=1.455265968 podStartE2EDuration="5.135228471s" totalImagesPullingTime="3.679962503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 02:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 02:53:54.133980963 +0000 UTC m=+377.929329528" watchObservedRunningTime="2026-11-02 02:53:54.135228471 +0000 UTC m=+377.930577036"
Nov 02 02:53:55.320716 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:55.320684    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:55.479526 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:55.479493    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:55.641378 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:55.641301    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:55.793632 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:55.793599    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:55.946387 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:55.946311    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:56.101281 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:56.101246    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:56.253764 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:56.253731    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:56.420012 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:56.419976    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:53:56.579808 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:53:56.579726    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:54:04.798929 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:54:04.798901    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-qj42d"
Nov 02 02:54:04.800664 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:54:04.800648    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-rhpmb"
Nov 02 02:54:05.188142 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:54:05.188049    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:54:24.801430 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:54:24.801354    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-kh22l"
Nov 02 02:54:30.292184 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:54:30.292139    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:54:35.187988 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:54:35.187951    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:55:05.188035 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:55:05.187998    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:55:08.068750 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:55:08.068715    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:55:08.226369 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:55:08.226337    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:55:16.041086 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:55:16.041043    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:55:30.682601 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:55:30.682563    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:55:35.188058 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:55:35.188020    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:56:05.188072 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:56:05.188034    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:56:28.538175 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:56:28.538132    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:56:30.977346 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:56:30.977310    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:56:34.495915 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:56:34.495880    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:56:34.667064 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:56:34.667027    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:56:35.188051 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:56:35.188012    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:57:05.187910 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:57:05.187874    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:57:05.574440 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:57:05.574408    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:57:07.266191 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:57:07.266115    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:57:31.328132 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:57:31.328049    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:57:35.190037 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:57:35.189995    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:58:05.187367 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:58:05.187326    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:58:08.388432 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:58:08.388402    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:58:08.576644 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:58:08.576598    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:58:31.664407 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:58:31.664370    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:58:35.188068 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:58:35.188028    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:59:05.187752 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:59:05.187717    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 02:59:20.741649 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:59:20.741604    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:59:20.909864 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:59:20.909827    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:59:32.013392 ip-10-0-131-112 kubenswrapper[4407]: I1102 02:59:32.013358    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 02:59:35.188267 ip-10-0-131-112 kubenswrapper[4407]: E1102 02:59:35.188231    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:00:05.187492 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:00:05.187456    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:00:32.358582 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:00:32.358507    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:00:35.188549 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:00:35.188513    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:01:05.187889 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:01:05.187854    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:01:32.709914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:01:32.709877    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:01:35.188847 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:01:35.188808    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:02:05.187127 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:02:05.187087    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:02:33.051734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:02:33.051700    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:02:35.187883 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:02:35.187847    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:02:59.448825 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:02:59.448792    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:05.187974 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:03:05.187941    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:03:09.893782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:09.893749    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:10.117083 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:10.117027    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:10.287752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:10.287721    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:10.447254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:10.447225    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:13.027340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.027308    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:03:13.030788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.030768    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:13.086323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.086296    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 02 03:03:13.136998 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.136961    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5fwnq\""
Nov 02 03:03:13.138219 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.138195    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:03:13.152302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.152262    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:13.179938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.179911    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4znf\" (UniqueName: \"kubernetes.io/projected/02205620-98a5-4f95-9761-0b0461a7ed68-kube-api-access-f4znf\") pod \"limitador-limitador-7c9b87d574-p4qfk\" (UID: \"02205620-98a5-4f95-9761-0b0461a7ed68\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:13.180084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.179945    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02205620-98a5-4f95-9761-0b0461a7ed68-config-file\") pod \"limitador-limitador-7c9b87d574-p4qfk\" (UID: \"02205620-98a5-4f95-9761-0b0461a7ed68\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:13.280791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.280719    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4znf\" (UniqueName: \"kubernetes.io/projected/02205620-98a5-4f95-9761-0b0461a7ed68-kube-api-access-f4znf\") pod \"limitador-limitador-7c9b87d574-p4qfk\" (UID: \"02205620-98a5-4f95-9761-0b0461a7ed68\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:13.280940 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.280802    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02205620-98a5-4f95-9761-0b0461a7ed68-config-file\") pod \"limitador-limitador-7c9b87d574-p4qfk\" (UID: \"02205620-98a5-4f95-9761-0b0461a7ed68\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:13.281359 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.281343    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02205620-98a5-4f95-9761-0b0461a7ed68-config-file\") pod \"limitador-limitador-7c9b87d574-p4qfk\" (UID: \"02205620-98a5-4f95-9761-0b0461a7ed68\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:13.327978 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.327942    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:03:13.329806 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.329785    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:13.338665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.338618    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4znf\" (UniqueName: \"kubernetes.io/projected/02205620-98a5-4f95-9761-0b0461a7ed68-kube-api-access-f4znf\") pod \"limitador-limitador-7c9b87d574-p4qfk\" (UID: \"02205620-98a5-4f95-9761-0b0461a7ed68\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:13.343479 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.343464    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:13.712737 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.712714    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:03:13.714603 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:03:13.714574    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02205620_98a5_4f95_9761_0b0461a7ed68.slice/crio-11911dfaa341fe2973580722f1497b3f89d925855f2ed1ee7cdc2554c5da7403 WatchSource:0}: Error finding container 11911dfaa341fe2973580722f1497b3f89d925855f2ed1ee7cdc2554c5da7403: Status 404 returned error can't find the container with id 11911dfaa341fe2973580722f1497b3f89d925855f2ed1ee7cdc2554c5da7403
Nov 02 03:03:13.716543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:13.716523    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 03:03:14.056936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:14.056903    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk" event={"ID":"02205620-98a5-4f95-9761-0b0461a7ed68","Type":"ContainerStarted","Data":"11911dfaa341fe2973580722f1497b3f89d925855f2ed1ee7cdc2554c5da7403"}
Nov 02 03:03:16.074376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:16.074342    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk" event={"ID":"02205620-98a5-4f95-9761-0b0461a7ed68","Type":"ContainerStarted","Data":"d1262c7eaa6d59b4f527fba5be28aea9c2efb06199a3030cbade8ec03a834eac"}
Nov 02 03:03:16.074853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:16.074484    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:16.190792 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:16.190739    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk" podStartSLOduration=2.470227995 podStartE2EDuration="4.190724962s" totalImagesPullingTime="1.720496967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:03:16.18936951 +0000 UTC m=+939.984718079" watchObservedRunningTime="2026-11-02 03:03:16.190724962 +0000 UTC m=+939.986073528"
Nov 02 03:03:27.080411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:27.080336    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"
Nov 02 03:03:33.306508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:33.306449    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:33.490821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:33.490773    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:34.015165 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:34.015135    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:35.187999 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:03:35.187965    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:03:51.641331 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:51.641295    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:54.861835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:54.861803    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:55.029529 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:55.029490    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:55.203878 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:55.203800    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:03:55.357564 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:03:55.357522    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:03.554787 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:03.554758    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:03.719237 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:03.719204    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:05.187839 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:04:05.187806    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:04:07.020639 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:07.020585    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:07.253338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:07.253280    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:33.738179 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:33.738145    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:35.188169 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:04:35.188129    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:04:37.229955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:37.229882    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:37.402212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:37.402177    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:39.334464 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:39.334411    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:39.523230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:39.523160    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:39.717050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:39.716938    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:45.861360 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:45.861323    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:04:54.976782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:54.976742    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:55.187959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.187927    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:04:55.439932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.439902    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"]
Nov 02 03:04:55.442965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.442947    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.543774 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.543738    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58066e0-8b2a-4916-9895-f88e667a8237-openshift-service-ca\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.543774 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.543774    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js6jj\" (UniqueName: \"kubernetes.io/projected/f58066e0-8b2a-4916-9895-f88e667a8237-kube-api-access-js6jj\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.543997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.543931    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f58066e0-8b2a-4916-9895-f88e667a8237-metrics-certs\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.544104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.544088    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f58066e0-8b2a-4916-9895-f88e667a8237-maas-api-tls\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.546869 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.546846    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 03:04:55.547007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.546912    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 03:04:55.547007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.546978    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 03:04:55.547111 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.547005    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pz7wg\""
Nov 02 03:04:55.547111 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.547059    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 02 03:04:55.548453 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.548428    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"]
Nov 02 03:04:55.645207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.645162    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f58066e0-8b2a-4916-9895-f88e667a8237-metrics-certs\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.645417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.645254    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f58066e0-8b2a-4916-9895-f88e667a8237-maas-api-tls\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.645417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.645286    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58066e0-8b2a-4916-9895-f88e667a8237-openshift-service-ca\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.645417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.645309    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6jj\" (UniqueName: \"kubernetes.io/projected/f58066e0-8b2a-4916-9895-f88e667a8237-kube-api-access-js6jj\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.646162 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.646140    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58066e0-8b2a-4916-9895-f88e667a8237-openshift-service-ca\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.647921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.647898    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f58066e0-8b2a-4916-9895-f88e667a8237-metrics-certs\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.648037 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.648017    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f58066e0-8b2a-4916-9895-f88e667a8237-maas-api-tls\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.704365 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.704295    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js6jj\" (UniqueName: \"kubernetes.io/projected/f58066e0-8b2a-4916-9895-f88e667a8237-kube-api-access-js6jj\") pod \"maas-api-57dc95b55c-tm7zm\" (UID: \"f58066e0-8b2a-4916-9895-f88e667a8237\") " pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:55.754874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:55.754843    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:56.127439 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:56.127412    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"]
Nov 02 03:04:56.129661 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:04:56.129618    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58066e0_8b2a_4916_9895_f88e667a8237.slice/crio-e835abc9338e50e64c6beeb406d371cac848e974dd1badb3649b3c8f85650dec WatchSource:0}: Error finding container e835abc9338e50e64c6beeb406d371cac848e974dd1badb3649b3c8f85650dec: Status 404 returned error can't find the container with id e835abc9338e50e64c6beeb406d371cac848e974dd1badb3649b3c8f85650dec
Nov 02 03:04:56.480659 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:56.480561    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm" event={"ID":"f58066e0-8b2a-4916-9895-f88e667a8237","Type":"ContainerStarted","Data":"e835abc9338e50e64c6beeb406d371cac848e974dd1badb3649b3c8f85650dec"}
Nov 02 03:04:59.500098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:59.500064    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm" event={"ID":"f58066e0-8b2a-4916-9895-f88e667a8237","Type":"ContainerStarted","Data":"9862fb39fb642e90cde976e2c4fa47af937671e2c0973dd0699f59e94a1a28b5"}
Nov 02 03:04:59.500472 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:59.500156    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:04:59.619828 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:04:59.619774    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm" podStartSLOduration=1.7484900190000001 podStartE2EDuration="4.619760404s" totalImagesPullingTime="2.871270385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:04:59.617908064 +0000 UTC m=+1043.413256642" watchObservedRunningTime="2026-11-02 03:04:59.619760404 +0000 UTC m=+1043.415108947"
Nov 02 03:05:05.187852 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:05:05.187811    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:05:05.509872 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:05:05.509839    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57dc95b55c-tm7zm"
Nov 02 03:05:34.077396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:05:34.077364    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:05:35.188187 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:05:35.188151    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:05:36.805033 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:05:36.804998    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:05:36.975011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:05:36.974980    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:05:44.409107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:05:44.409071    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:05:44.569971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:05:44.569934    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:05.188330 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:06:05.188284    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:06:06.928484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:06.928449    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:07.391561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:07.391528    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:08.565665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:08.565608    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:08.732230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:08.732186    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:14.484245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.484205    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"]
Nov 02 03:06:14.487822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.487799    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.593595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.593556    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 02 03:06:14.593793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.593648    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 02 03:06:14.593793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.593664    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 02 03:06:14.593793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.593653    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-n5h79\""
Nov 02 03:06:14.595066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.595047    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"]
Nov 02 03:06:14.612298 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.612275    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:14.639423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.639394    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.639543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.639431    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.639543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.639452    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.639661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.639578    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.639661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.639644    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b79df1c-7f4d-4e26-bd41-157a2e656c23-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.639775 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.639739    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbkfb\" (UniqueName: \"kubernetes.io/projected/9b79df1c-7f4d-4e26-bd41-157a2e656c23-kube-api-access-jbkfb\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740288    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740333    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740370    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740413    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740451    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b79df1c-7f4d-4e26-bd41-157a2e656c23-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740476    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbkfb\" (UniqueName: \"kubernetes.io/projected/9b79df1c-7f4d-4e26-bd41-157a2e656c23-kube-api-access-jbkfb\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740779    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740836    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.740931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.740859    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.742801 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.742778    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b79df1c-7f4d-4e26-bd41-157a2e656c23-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.743055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.743038    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b79df1c-7f4d-4e26-bd41-157a2e656c23-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.798754 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.798720    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbkfb\" (UniqueName: \"kubernetes.io/projected/9b79df1c-7f4d-4e26-bd41-157a2e656c23-kube-api-access-jbkfb\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj\" (UID: \"9b79df1c-7f4d-4e26-bd41-157a2e656c23\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:14.799898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:14.799863    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:15.169757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:15.169734    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"]
Nov 02 03:06:15.171541 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:06:15.171515    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b79df1c_7f4d_4e26_bd41_157a2e656c23.slice/crio-4c0d7e864cc6291f9dfdc5a6d9ce1a0cbfa3e76be98143e203aa3ef492ae1eb6 WatchSource:0}: Error finding container 4c0d7e864cc6291f9dfdc5a6d9ce1a0cbfa3e76be98143e203aa3ef492ae1eb6: Status 404 returned error can't find the container with id 4c0d7e864cc6291f9dfdc5a6d9ce1a0cbfa3e76be98143e203aa3ef492ae1eb6
Nov 02 03:06:15.814373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:15.814325    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj" event={"ID":"9b79df1c-7f4d-4e26-bd41-157a2e656c23","Type":"ContainerStarted","Data":"4c0d7e864cc6291f9dfdc5a6d9ce1a0cbfa3e76be98143e203aa3ef492ae1eb6"}
Nov 02 03:06:17.826758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:17.826722    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj" event={"ID":"9b79df1c-7f4d-4e26-bd41-157a2e656c23","Type":"ContainerStarted","Data":"1d3b34dcdcccb525e715d01e0488e84c4416e60bb63c5c15db61bf39beea4918"}
Nov 02 03:06:17.943126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:17.943030    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj" podStartSLOduration=1.730094536 podStartE2EDuration="3.943016406s" totalImagesPullingTime="2.21292187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:06:17.941969164 +0000 UTC m=+1121.737317729" watchObservedRunningTime="2026-11-02 03:06:17.943016406 +0000 UTC m=+1121.738364976"
Nov 02 03:06:18.209048 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:18.208964    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:06:18.800632 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:18.800600    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:21.835347 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:21.835311    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:24.800069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:24.800039    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:24.812334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:24.812309    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:24.872597 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:24.872566    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj"
Nov 02 03:06:25.378841 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:25.378792    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:25.451132 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:25.451100    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:06:29.285533 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:29.285486    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:32.512658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.509973    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"]
Nov 02 03:06:32.514747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.514728    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.588409 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.588369    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.588567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.588419    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.588567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.588520    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.588680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.588561    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1c469a1-a23a-47df-8dbc-8d9e642efc40-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.588720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.588684    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.588761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.588738    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv4h9\" (UniqueName: \"kubernetes.io/projected/a1c469a1-a23a-47df-8dbc-8d9e642efc40-kube-api-access-tv4h9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.620206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.620181    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 02 03:06:32.622171 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.622151    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"]
Nov 02 03:06:32.628152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.628131    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:32.689511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.689478    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.689704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.689526    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv4h9\" (UniqueName: \"kubernetes.io/projected/a1c469a1-a23a-47df-8dbc-8d9e642efc40-kube-api-access-tv4h9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.689704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.689552    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.689704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.689572    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.689704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.689606    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.689704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.689665    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1c469a1-a23a-47df-8dbc-8d9e642efc40-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.689994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.689969    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.690104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.690012    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.690104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.690079    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.692038 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.692011    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1c469a1-a23a-47df-8dbc-8d9e642efc40-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.692181 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.692164    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1c469a1-a23a-47df-8dbc-8d9e642efc40-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.747613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.747578    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv4h9\" (UniqueName: \"kubernetes.io/projected/a1c469a1-a23a-47df-8dbc-8d9e642efc40-kube-api-access-tv4h9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv\" (UID: \"a1c469a1-a23a-47df-8dbc-8d9e642efc40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:32.827572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:32.827503    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:33.200070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:33.200046    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"]
Nov 02 03:06:33.200740 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:06:33.200716    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c469a1_a23a_47df_8dbc_8d9e642efc40.slice/crio-0318520f6d35674cf6681430d06c96898cd686771051972ace7df047b5b0bf2e WatchSource:0}: Error finding container 0318520f6d35674cf6681430d06c96898cd686771051972ace7df047b5b0bf2e: Status 404 returned error can't find the container with id 0318520f6d35674cf6681430d06c96898cd686771051972ace7df047b5b0bf2e
Nov 02 03:06:33.903371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:33.903340    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv" event={"ID":"a1c469a1-a23a-47df-8dbc-8d9e642efc40","Type":"ContainerStarted","Data":"0b1e6e8039cdfde86a65e15a61d7ec070568049bf58053761e6d27cac71d87eb"}
Nov 02 03:06:33.903768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:33.903389    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv" event={"ID":"a1c469a1-a23a-47df-8dbc-8d9e642efc40","Type":"ContainerStarted","Data":"0318520f6d35674cf6681430d06c96898cd686771051972ace7df047b5b0bf2e"}
Nov 02 03:06:34.020323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:34.020273    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv" podStartSLOduration=1.445237881 podStartE2EDuration="2.020257515s" totalImagesPullingTime="575.019634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:06:34.018496428 +0000 UTC m=+1137.813844993" watchObservedRunningTime="2026-11-02 03:06:34.020257515 +0000 UTC m=+1137.815606110"
Nov 02 03:06:34.413394 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:34.413361    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:34.828249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:34.828218    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:35.187751 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:06:35.187662    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:06:35.363996 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:35.363962    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:06:36.455068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.455043    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-hxn75"]
Nov 02 03:06:36.458436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.458419    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.516317 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.516287    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-hxn75"]
Nov 02 03:06:36.525735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.525694    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-home\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.525907 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.525764    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smnw4\" (UniqueName: \"kubernetes.io/projected/6be96fa7-288b-40a2-8507-5b4da7da07a7-kube-api-access-smnw4\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.525907 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.525805    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.525907 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.525827    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.526071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.525969    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6be96fa7-288b-40a2-8507-5b4da7da07a7-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.526071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.526003    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.564910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.564879    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 02 03:06:36.570413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.570387    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:36.626479 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.626450    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smnw4\" (UniqueName: \"kubernetes.io/projected/6be96fa7-288b-40a2-8507-5b4da7da07a7-kube-api-access-smnw4\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.626653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.626485    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.626653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.626502    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.626653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.626540    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6be96fa7-288b-40a2-8507-5b4da7da07a7-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.626653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.626567    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.626653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.626608    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-home\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.627056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.627034    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.627127 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.627051    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.627127 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.627098    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-home\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.628895 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.628879    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6be96fa7-288b-40a2-8507-5b4da7da07a7-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.629253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.629232    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6be96fa7-288b-40a2-8507-5b4da7da07a7-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.686845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.686810    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smnw4\" (UniqueName: \"kubernetes.io/projected/6be96fa7-288b-40a2-8507-5b4da7da07a7-kube-api-access-smnw4\") pod \"echo-simulator-kserve-64b6f8f96c-hxn75\" (UID: \"6be96fa7-288b-40a2-8507-5b4da7da07a7\") " pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:36.770973 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:36.770936    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:37.140073 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:37.140048    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-hxn75"]
Nov 02 03:06:37.141346 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:06:37.141318    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be96fa7_288b_40a2_8507_5b4da7da07a7.slice/crio-1eccab9556f8f08ae8dfbd68fb5d9f57fbffaa2f7a976b0ca30c8d1efd437741 WatchSource:0}: Error finding container 1eccab9556f8f08ae8dfbd68fb5d9f57fbffaa2f7a976b0ca30c8d1efd437741: Status 404 returned error can't find the container with id 1eccab9556f8f08ae8dfbd68fb5d9f57fbffaa2f7a976b0ca30c8d1efd437741
Nov 02 03:06:37.922506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:37.922468    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75" event={"ID":"6be96fa7-288b-40a2-8507-5b4da7da07a7","Type":"ContainerStarted","Data":"1eccab9556f8f08ae8dfbd68fb5d9f57fbffaa2f7a976b0ca30c8d1efd437741"}
Nov 02 03:06:39.279539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.279506    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"]
Nov 02 03:06:39.283192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.283169    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.354340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.354305    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.354491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.354353    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74tpk\" (UniqueName: \"kubernetes.io/projected/7cd159c5-459e-4cda-8d90-f9264901044b-kube-api-access-74tpk\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.354491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.354386    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.354491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.354459    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.354491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.354486    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.354652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.354517    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd159c5-459e-4cda-8d90-f9264901044b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.361260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.361232    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 02 03:06:39.390676 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.390617    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"]
Nov 02 03:06:39.396562 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.396519    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:39.455027 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.454998    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.455027 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.455028    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74tpk\" (UniqueName: \"kubernetes.io/projected/7cd159c5-459e-4cda-8d90-f9264901044b-kube-api-access-74tpk\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.455229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.455049    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.455229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.455089    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.455229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.455103    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.455229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.455121    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd159c5-459e-4cda-8d90-f9264901044b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.455449 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.455413    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.455525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.455502    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.455561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.455529    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.457516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.457495    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd159c5-459e-4cda-8d90-f9264901044b-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.457680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.457590    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd159c5-459e-4cda-8d90-f9264901044b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.512499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.512469    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74tpk\" (UniqueName: \"kubernetes.io/projected/7cd159c5-459e-4cda-8d90-f9264901044b-kube-api-access-74tpk\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s\" (UID: \"7cd159c5-459e-4cda-8d90-f9264901044b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.600257 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.600213    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:39.932285 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.932193    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75" event={"ID":"6be96fa7-288b-40a2-8507-5b4da7da07a7","Type":"ContainerStarted","Data":"ffa2e1b071c10810b5f602ff84f4af549a2d10a6b1888a3317d4b8e646d7a29f"}
Nov 02 03:06:39.969906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:39.969885    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"]
Nov 02 03:06:39.971469 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:06:39.971443    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd159c5_459e_4cda_8d90_f9264901044b.slice/crio-abd57748c95acc8566d9b81743f957ad3f5bc1175c7117ac950d7bbcabfc5250 WatchSource:0}: Error finding container abd57748c95acc8566d9b81743f957ad3f5bc1175c7117ac950d7bbcabfc5250: Status 404 returned error can't find the container with id abd57748c95acc8566d9b81743f957ad3f5bc1175c7117ac950d7bbcabfc5250
Nov 02 03:06:40.046219 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:40.046169    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75" podStartSLOduration=1.677449846 podStartE2EDuration="4.046154485s" totalImagesPullingTime="2.368704639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:06:40.046043933 +0000 UTC m=+1143.841392499" watchObservedRunningTime="2026-11-02 03:06:40.046154485 +0000 UTC m=+1143.841503049"
Nov 02 03:06:40.771557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:40.771517    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:40.938189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:40.938085    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s" event={"ID":"7cd159c5-459e-4cda-8d90-f9264901044b","Type":"ContainerStarted","Data":"f1a3612b8d5c97bd60f88e8474735e93fb1e7343bfa7a1a237a9224b0dec0be9"}
Nov 02 03:06:40.938189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:40.938149    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s" event={"ID":"7cd159c5-459e-4cda-8d90-f9264901044b","Type":"ContainerStarted","Data":"abd57748c95acc8566d9b81743f957ad3f5bc1175c7117ac950d7bbcabfc5250"}
Nov 02 03:06:41.055583 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:41.055536    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s" podStartSLOduration=1.470628274 podStartE2EDuration="2.055522381s" totalImagesPullingTime="584.894107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:06:41.054063408 +0000 UTC m=+1144.849411974" watchObservedRunningTime="2026-11-02 03:06:41.055522381 +0000 UTC m=+1144.850870945"
Nov 02 03:06:41.601224 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:41.601189    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:42.136846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:42.136806    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:42.828147 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:42.828113    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:42.840840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:42.840815    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:42.957741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:42.957712    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv"
Nov 02 03:06:43.268804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:43.268742    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:06:46.771888 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:46.771856    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:46.784238 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:46.784213    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:46.974220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:46.974195    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-hxn75"
Nov 02 03:06:47.679281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.679247    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"]
Nov 02 03:06:47.682880 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.682863    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.740561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.740532    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tk8sg\""
Nov 02 03:06:47.787678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.787648    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 02 03:06:47.789827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.789800    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"]
Nov 02 03:06:47.829209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829182    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.829367 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829218    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x89n4\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-kube-api-access-x89n4\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.829367 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829238    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.829437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829362    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.829437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829399    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.829437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829420    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.829529 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829507    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.829562 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829548    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.829602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.829573    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b6aabd-8241-47d5-93f2-494bfcef0106-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.930944    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.930978    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b6aabd-8241-47d5-93f2-494bfcef0106-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931022    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931051    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x89n4\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-kube-api-access-x89n4\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931077    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931146    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931188    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931233    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931259    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931433    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931604    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.931740 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.931716    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.932089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.932063    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b6aabd-8241-47d5-93f2-494bfcef0106-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.932196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.932062    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.933434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.933411    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.934031 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.934005    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.989195 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.989161    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.989323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.989282    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x89n4\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-kube-api-access-x89n4\") pod \"maas-default-gateway-openshift-default-58847f9fc5-gnf96\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:47.995049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:47.995025    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:48.416183 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:48.416161    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"]
Nov 02 03:06:48.418308 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:06:48.418281    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b6aabd_8241_47d5_93f2_494bfcef0106.slice/crio-56466993b3905767684d366f02f19151617d115a0baa93c69c7547cd9ca50d92 WatchSource:0}: Error finding container 56466993b3905767684d366f02f19151617d115a0baa93c69c7547cd9ca50d92: Status 404 returned error can't find the container with id 56466993b3905767684d366f02f19151617d115a0baa93c69c7547cd9ca50d92
Nov 02 03:06:48.973570 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:48.973542    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" event={"ID":"14b6aabd-8241-47d5-93f2-494bfcef0106","Type":"ContainerStarted","Data":"56466993b3905767684d366f02f19151617d115a0baa93c69c7547cd9ca50d92"}
Nov 02 03:06:49.600431 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:49.600394    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:49.612484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:49.612462    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:49.989063 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:49.988984    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s"
Nov 02 03:06:50.061446 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:50.061414    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:06:52.478983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:52.478954    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:06:52.479231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:52.479018    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:06:52.479231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:52.479048    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:06:53.012428 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:53.012388    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" event={"ID":"14b6aabd-8241-47d5-93f2-494bfcef0106","Type":"ContainerStarted","Data":"467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528"}
Nov 02 03:06:53.231270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:53.231217    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podStartSLOduration=2.173298022 podStartE2EDuration="6.23119872s" totalImagesPullingTime="4.057900698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:06:53.179377254 +0000 UTC m=+1156.974725820" watchObservedRunningTime="2026-11-02 03:06:53.23119872 +0000 UTC m=+1157.026547287"
Nov 02 03:06:53.997656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:53.997268    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:06:54.007096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:54.007058    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-gnf96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 03:06:54.007266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:54.007133    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 03:06:55.000647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:54.999605    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-gnf96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 03:06:55.000647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:54.999744    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 03:06:56.001119 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:56.001085    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-gnf96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 03:06:56.001761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:56.001728    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 03:06:57.000648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:57.000531    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-gnf96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 03:06:57.000648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:57.000602    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 03:06:57.999764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:57.999302    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-gnf96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 03:06:57.999764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:57.999372    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 03:06:58.995685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:58.995652    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-gnf96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 02 03:06:58.995883 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:58.995707    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 02 03:06:59.302949 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:06:59.302856    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:00.698193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:00.698156    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:07:00.698677 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:00.698563    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:07:00.699537 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:00.699517    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:07:05.188191 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:07:05.188143    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:07:12.591149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:12.591112    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:12.752843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:12.752810    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:14.044707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:14.044669    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:14.213994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:14.213964    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:34.757518 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:34.757484    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:35.187977 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:07:35.187890    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:07:43.049152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:43.049120    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:43.215715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:43.215677    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:43.375904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:43.375819    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:43.556871 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:43.556843    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:43.888495 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:43.888449    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:44.018704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.018674    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"]
Nov 02 03:07:44.022121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.022100    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.057185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.057144    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:44.075016 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.074984    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-66a47a3b-openshift-default-dockercfg-8gvjv\""
Nov 02 03:07:44.127416 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.127370    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"]
Nov 02 03:07:44.192690 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.192593    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwb24\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-kube-api-access-vwb24\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.192690 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.192645    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-credential-socket\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.192892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.192738    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-podinfo\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.192892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.192776    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-envoy\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.192892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.192877    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istiod-ca-cert\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.193007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.192899    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-certs\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.193007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.192920    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-data\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.193071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.193032    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-socket\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.193071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.193061    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-token\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.227325 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.227279    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:44.293531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293506    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-certs\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.293751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293539    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-data\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.293751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293576    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-socket\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.293751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293605    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-token\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.293751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293714    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwb24\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-kube-api-access-vwb24\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.293981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293767    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-credential-socket\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.293981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293805    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-podinfo\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.293981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293842    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-envoy\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.293981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293930    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istiod-ca-cert\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.294128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.293993    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-certs\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.294128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.294075    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-data\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.294198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.294134    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-socket\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.294242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.294223    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-credential-socket\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.294547 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.294531    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istiod-ca-cert\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.296329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.296309    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-envoy\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.296537 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.296519    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-podinfo\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.351617 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.351585    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwb24\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-kube-api-access-vwb24\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.351881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.351862    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-token\") pod \"e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") " pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.428394 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.428355    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:44.625614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.625568    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"]
Nov 02 03:07:44.630198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.630178    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.634732 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.634710    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:44.689231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.689196    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"]
Nov 02 03:07:44.689416 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.689266    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-935e4abf-gw-openshift-default-dockercfg-xn7mx\""
Nov 02 03:07:44.740962 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.740934    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:44.798656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.798606    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-podinfo\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.798836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.798707    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-data\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.798836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.798748    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-certs\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.798836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.798794    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-token\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.798988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.798841    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-envoy\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.798988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.798909    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istiod-ca-cert\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.798988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.798938    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-socket\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.799136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.799053    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-credential-socket\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.799136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.799078    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6r57\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-kube-api-access-x6r57\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.899929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.899815    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-credential-socket\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.899929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.899856    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6r57\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-kube-api-access-x6r57\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.899929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.899916    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-podinfo\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.899943    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-data\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.899974    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-certs\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.900024    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-token\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.900052    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-envoy\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.900083    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istiod-ca-cert\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.900107    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-socket\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.900349    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-credential-socket\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.900497    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-socket\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.900663    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-certs\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.900977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.900938    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-data\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.901210 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.901187    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istiod-ca-cert\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.903149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.903128    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-envoy\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.903348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.903323    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:44.903460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.903445    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-podinfo\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.961146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.961117    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-token\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:44.961320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:44.961264    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6r57\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-kube-api-access-x6r57\") pod \"e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") " pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:45.064085 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.064052    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:45.070939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.070912    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"]
Nov 02 03:07:45.072708 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:07:45.072685    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d554f51_8f1d_46e8_9703_f7e4e5578ac4.slice/crio-0297db6024a162e4958fb9657456487230cab1a95e76500e00ef5746c74f6ac8 WatchSource:0}: Error finding container 0297db6024a162e4958fb9657456487230cab1a95e76500e00ef5746c74f6ac8: Status 404 returned error can't find the container with id 0297db6024a162e4958fb9657456487230cab1a95e76500e00ef5746c74f6ac8
Nov 02 03:07:45.074869 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.074838    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:45.074926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.074902    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:45.074961 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.074931    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:45.244487 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.244463    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:45.265364 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.265337    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" event={"ID":"9d554f51-8f1d-46e8-9703-f7e4e5578ac4","Type":"ContainerStarted","Data":"86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d"}
Nov 02 03:07:45.265499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.265396    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" event={"ID":"9d554f51-8f1d-46e8-9703-f7e4e5578ac4","Type":"ContainerStarted","Data":"0297db6024a162e4958fb9657456487230cab1a95e76500e00ef5746c74f6ac8"}
Nov 02 03:07:45.388866 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.388818    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" podStartSLOduration=2.388802052 podStartE2EDuration="2.388802052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:07:45.387115251 +0000 UTC m=+1209.182463815" watchObservedRunningTime="2026-11-02 03:07:45.388802052 +0000 UTC m=+1209.184150617"
Nov 02 03:07:45.635909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.635825    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:45.669154 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.669113    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"]
Nov 02 03:07:45.670669 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:07:45.670639    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dffca2b_9932_4960_b2e5_3c5f67fef0be.slice/crio-44bc5f6d360f11fbb9b0b2e78822527bbae249f57432aaa234da28bf90b76781 WatchSource:0}: Error finding container 44bc5f6d360f11fbb9b0b2e78822527bbae249f57432aaa234da28bf90b76781: Status 404 returned error can't find the container with id 44bc5f6d360f11fbb9b0b2e78822527bbae249f57432aaa234da28bf90b76781
Nov 02 03:07:45.673243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.673213    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:45.673310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.673282    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:45.673347 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:45.673314    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:46.098997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.098969    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"]
Nov 02 03:07:46.102489 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.102471    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.158745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.158716    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4bbf98be-openshift-default-dockercfg-gc4vq\""
Nov 02 03:07:46.160790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.160767    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"]
Nov 02 03:07:46.212586 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.212556    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-envoy\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.212778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.212592    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcph5\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-kube-api-access-vcph5\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.212778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.212697    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-socket\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.212778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.212769    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-istio-token\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.212896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.212823    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-credential-socket\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.212971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.212940    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-certs\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.213009 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.212994    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/303d7d7c-516b-4fee-8212-3bab14188a08-istio-podinfo\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.213053 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.213021    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/303d7d7c-516b-4fee-8212-3bab14188a08-istiod-ca-cert\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.213111 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.213094    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-data\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.222834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.222810    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:46.272328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.272294    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" event={"ID":"9dffca2b-9932-4960-b2e5-3c5f67fef0be","Type":"ContainerStarted","Data":"67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6"}
Nov 02 03:07:46.272510 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.272349    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" event={"ID":"9dffca2b-9932-4960-b2e5-3c5f67fef0be","Type":"ContainerStarted","Data":"44bc5f6d360f11fbb9b0b2e78822527bbae249f57432aaa234da28bf90b76781"}
Nov 02 03:07:46.313830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.313778    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/303d7d7c-516b-4fee-8212-3bab14188a08-istio-podinfo\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.313830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.313832    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/303d7d7c-516b-4fee-8212-3bab14188a08-istiod-ca-cert\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.313858    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-data\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.313923    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-envoy\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314034    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcph5\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-kube-api-access-vcph5\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314106    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-socket\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314144    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-istio-token\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314201    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-credential-socket\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314243    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-certs\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314426 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314367    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-data\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314600    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-certs\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314647    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-credential-socket\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314705    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-socket\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.314768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.314763    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/303d7d7c-516b-4fee-8212-3bab14188a08-istiod-ca-cert\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.316335 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.316315    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-envoy\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.316335 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.316331    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/303d7d7c-516b-4fee-8212-3bab14188a08-istio-podinfo\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.372393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.372305    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcph5\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-kube-api-access-vcph5\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.372393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.372310    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-istio-token\") pod \"e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") " pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.393309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.393263    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" podStartSLOduration=2.393248263 podStartE2EDuration="2.393248263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:07:46.392051307 +0000 UTC m=+1210.187399884" watchObservedRunningTime="2026-11-02 03:07:46.393248263 +0000 UTC m=+1210.188596828"
Nov 02 03:07:46.415415 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.415385    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:46.440596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.440538    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:46.640760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.640690    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:46.836086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.836065    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"]
Nov 02 03:07:46.837666 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:07:46.837638    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod303d7d7c_516b_4fee_8212_3bab14188a08.slice/crio-94a2d452e880b8b6ce764532f20e0b3b4d5e7f121fe4a1a83e8b82c7efd07673 WatchSource:0}: Error finding container 94a2d452e880b8b6ce764532f20e0b3b4d5e7f121fe4a1a83e8b82c7efd07673: Status 404 returned error can't find the container with id 94a2d452e880b8b6ce764532f20e0b3b4d5e7f121fe4a1a83e8b82c7efd07673
Nov 02 03:07:46.839859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.839831    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:46.839925 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.839891    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:46.839925 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:46.839918    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:47.244778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.244749    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:47.249562 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.249540    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:47.277666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.277616    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" event={"ID":"303d7d7c-516b-4fee-8212-3bab14188a08","Type":"ContainerStarted","Data":"96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7"}
Nov 02 03:07:47.277799 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.277690    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" event={"ID":"303d7d7c-516b-4fee-8212-3bab14188a08","Type":"ContainerStarted","Data":"94a2d452e880b8b6ce764532f20e0b3b4d5e7f121fe4a1a83e8b82c7efd07673"}
Nov 02 03:07:47.278158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.278084    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:47.278158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.278129    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:47.278687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.278668    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:07:47.278842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.278829    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:07:47.416287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.416254    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:47.421501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.421475    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:47.538465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.538388    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:47.601778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.601728    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" podStartSLOduration=1.601713884 podStartE2EDuration="1.601713884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:07:47.600738312 +0000 UTC m=+1211.396086877" watchObservedRunningTime="2026-11-02 03:07:47.601713884 +0000 UTC m=+1211.397062448"
Nov 02 03:07:47.714958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.714915    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:47.877663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:47.877571    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:48.241391 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:48.241289    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:48.282755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:48.282721    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:48.283707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:48.283689    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:07:48.417275 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:48.417241    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:49.182960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:49.182922    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:49.397342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:49.397311    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:49.798011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:49.797979    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:49.966763 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:49.966730    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:50.128348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.128275    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:50.569402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.569369    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"]
Nov 02 03:07:50.573200 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.573180    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.626069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.626038    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-bc460382-openshift-default-dockercfg-svlth\""
Nov 02 03:07:50.674682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.674651    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:50.678577 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.678554    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"]
Nov 02 03:07:50.761858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.761825    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-credential-socket\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.762030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.761870    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-token\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.762030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.761941    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8f8v\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-kube-api-access-d8f8v\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.762030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.761979    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-socket\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.762162 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.762092    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istiod-ca-cert\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.762162 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.762140    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-podinfo\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.762265 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.762246    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-envoy\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.762321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.762284    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-data\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.762378 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.762365    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-certs\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.862969    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istiod-ca-cert\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863018    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-podinfo\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863053    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-envoy\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863073    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-data\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863111    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-certs\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863145    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-credential-socket\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863170    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-token\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863187    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8f8v\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-kube-api-access-d8f8v\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863585 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863379    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-socket\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863585 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863509    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-data\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863804    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-socket\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.863914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.863821    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-certs\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.864033 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.864015    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-credential-socket\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.864082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.864042    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istiod-ca-cert\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.865580 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.865561    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-envoy\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.865675 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.865640    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-podinfo\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.867706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.867686    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:50.920679 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.920654    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-token\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:50.920824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:50.920749    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8f8v\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-kube-api-access-d8f8v\") pod \"e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") " pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:51.186244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:51.186161    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:51.606384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:51.606354    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"]
Nov 02 03:07:51.607896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:51.607875    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:51.645807 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:07:51.645782    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe5b198_f1b9_4d32_8e34_6faa1c22fc03.slice/crio-1b48f8c035b785f09b1fea5c89c2569c4ec19672fc1a58eb61df304c5c3b03c4 WatchSource:0}: Error finding container 1b48f8c035b785f09b1fea5c89c2569c4ec19672fc1a58eb61df304c5c3b03c4: Status 404 returned error can't find the container with id 1b48f8c035b785f09b1fea5c89c2569c4ec19672fc1a58eb61df304c5c3b03c4
Nov 02 03:07:51.648072 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:51.648043    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:51.648136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:51.648106    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:51.648181 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:51.648137    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:07:51.777837 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:51.777790    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:52.302288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:52.302254    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" event={"ID":"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03","Type":"ContainerStarted","Data":"10df0bd1694665d487af1eaccd2b7a48e3f5eebe1a5590fececd8cca0aa4ef33"}
Nov 02 03:07:52.302461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:52.302300    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" event={"ID":"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03","Type":"ContainerStarted","Data":"1b48f8c035b785f09b1fea5c89c2569c4ec19672fc1a58eb61df304c5c3b03c4"}
Nov 02 03:07:52.470109 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:52.469896    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" podStartSLOduration=2.469880567 podStartE2EDuration="2.469880567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:07:52.469833554 +0000 UTC m=+1216.265182119" watchObservedRunningTime="2026-11-02 03:07:52.469880567 +0000 UTC m=+1216.265229132"
Nov 02 03:07:53.187383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.187355    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:53.192199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.192175    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:53.307299 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.307273    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:53.308319 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.308303    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:07:53.646715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.646681    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"]
Nov 02 03:07:53.650267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.650252    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.688424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.688380    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:53.693555 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.693531    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brcgr\" (UniqueName: \"kubernetes.io/projected/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-kube-api-access-brcgr\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.693655 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.693590    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.693720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.693671    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-openshift-service-ca\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.693761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.693731    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-metrics-certs\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.705693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.705666    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d1107301-serving-cert\""
Nov 02 03:07:53.708341 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.708309    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"]
Nov 02 03:07:53.794766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.794737    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brcgr\" (UniqueName: \"kubernetes.io/projected/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-kube-api-access-brcgr\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.794959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.794817    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.794959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.794846    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-openshift-service-ca\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.795080 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.795028    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-metrics-certs\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.795399 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:07:53.795177    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-serving-cert: secret "maas-api-e2e-ait-d1107301-serving-cert" not found
Nov 02 03:07:53.795399 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:07:53.795245    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls podName:0d0bea4d-8d70-4e34-898d-0af0607e7f1f nodeName:}" failed. No retries permitted until 2026-11-02 03:07:54.29522364 +0000 UTC m=+1218.090572183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls") pod "maas-api-e2e-ait-d1107301-7fd646f6df-5sfph" (UID: "0d0bea4d-8d70-4e34-898d-0af0607e7f1f") : secret "maas-api-e2e-ait-d1107301-serving-cert" not found
Nov 02 03:07:53.795980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.795958    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-openshift-service-ca\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.798071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.798051    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-metrics-certs\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.855500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.855474    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brcgr\" (UniqueName: \"kubernetes.io/projected/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-kube-api-access-brcgr\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:53.874389 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:53.874363    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:54.037969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:54.037940    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:54.224394 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:54.224348    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:54.301108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:54.301024    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:54.303632 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:54.303588    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls\") pod \"maas-api-e2e-ait-d1107301-7fd646f6df-5sfph\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:54.563400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:54.563315    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:54.933546 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:54.933517    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"]
Nov 02 03:07:54.935083 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:07:54.935058    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0bea4d_8d70_4e34_898d_0af0607e7f1f.slice/crio-7ad766758f76d285f0abb39ecf3c9340eada516a84e954b472c3838c502f5ee5 WatchSource:0}: Error finding container 7ad766758f76d285f0abb39ecf3c9340eada516a84e954b472c3838c502f5ee5: Status 404 returned error can't find the container with id 7ad766758f76d285f0abb39ecf3c9340eada516a84e954b472c3838c502f5ee5
Nov 02 03:07:55.316892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:55.316856    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph" event={"ID":"0d0bea4d-8d70-4e34-898d-0af0607e7f1f","Type":"ContainerStarted","Data":"7ad766758f76d285f0abb39ecf3c9340eada516a84e954b472c3838c502f5ee5"}
Nov 02 03:07:56.323610 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:56.323558    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:56.325425 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:56.325398    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph" event={"ID":"0d0bea4d-8d70-4e34-898d-0af0607e7f1f","Type":"ContainerStarted","Data":"628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f"}
Nov 02 03:07:56.325552 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:56.325525    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:07:56.443540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:56.443488    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph" podStartSLOduration=2.411584552 podStartE2EDuration="3.443475872s" totalImagesPullingTime="1.03189132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:07:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:07:56.442327843 +0000 UTC m=+1220.237676408" watchObservedRunningTime="2026-11-02 03:07:56.443475872 +0000 UTC m=+1220.238824437"
Nov 02 03:07:56.485204 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:56.485153    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:58.515596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:58.515525    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:58.698139 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:58.698114    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:58.873712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:58.873682    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:59.139386 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:59.139250    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:59.311244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:59.311217    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:07:59.463285 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:07:59.463200    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:01.256760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:01.256713    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:01.440873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:01.440842    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:02.335347 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:02.335321    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:08:03.176253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:03.176226    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:08:04.257817 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:04.257787    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:05.188032 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:08:05.188000    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:08:09.208649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.208593    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"]
Nov 02 03:08:09.212395 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.212374    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.317123 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.317094    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-66a47a3b-serving-cert\""
Nov 02 03:08:09.318997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.318977    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"]
Nov 02 03:08:09.363245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.363214    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.363414 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.363258    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fdf10b4-8a98-4008-b344-21e6e839077a-openshift-service-ca\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.363414 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.363365    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-metrics-certs\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.363414 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.363408    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qth2b\" (UniqueName: \"kubernetes.io/projected/1fdf10b4-8a98-4008-b344-21e6e839077a-kube-api-access-qth2b\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.464808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.464734    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.464808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.464772    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fdf10b4-8a98-4008-b344-21e6e839077a-openshift-service-ca\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.465010 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.464816    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-metrics-certs\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.465010 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.464856    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qth2b\" (UniqueName: \"kubernetes.io/projected/1fdf10b4-8a98-4008-b344-21e6e839077a-kube-api-access-qth2b\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.465010 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:08:09.464878    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-serving-cert: secret "maas-api-e2e-ait-66a47a3b-serving-cert" not found
Nov 02 03:08:09.465010 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:08:09.464945    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls podName:1fdf10b4-8a98-4008-b344-21e6e839077a nodeName:}" failed. No retries permitted until 2026-11-02 03:08:09.964925554 +0000 UTC m=+1233.760274124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls") pod "maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg" (UID: "1fdf10b4-8a98-4008-b344-21e6e839077a") : secret "maas-api-e2e-ait-66a47a3b-serving-cert" not found
Nov 02 03:08:09.465438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.465419    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fdf10b4-8a98-4008-b344-21e6e839077a-openshift-service-ca\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.467293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.467276    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-metrics-certs\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.523068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.523036    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qth2b\" (UniqueName: \"kubernetes.io/projected/1fdf10b4-8a98-4008-b344-21e6e839077a-kube-api-access-qth2b\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.574884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.574838    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:09.740773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.740674    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:09.843169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.843139    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:08:09.885872 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.885831    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:09.970459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.970420    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:09.972890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:09.972868    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls\") pod \"maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:10.126416 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:10.126380    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:10.511904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:10.511516    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"]
Nov 02 03:08:10.513764 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:08:10.513739    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fdf10b4_8a98_4008_b344_21e6e839077a.slice/crio-89f0d840d6c8ef372437c710e079beda869e507aae3a3c03028c58fb569e0845 WatchSource:0}: Error finding container 89f0d840d6c8ef372437c710e079beda869e507aae3a3c03028c58fb569e0845: Status 404 returned error can't find the container with id 89f0d840d6c8ef372437c710e079beda869e507aae3a3c03028c58fb569e0845
Nov 02 03:08:10.559569 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:10.559539    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:10.722862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:10.722835    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:11.411234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:11.411205    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg" event={"ID":"1fdf10b4-8a98-4008-b344-21e6e839077a","Type":"ContainerStarted","Data":"89f0d840d6c8ef372437c710e079beda869e507aae3a3c03028c58fb569e0845"}
Nov 02 03:08:12.417670 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:12.417609    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg" event={"ID":"1fdf10b4-8a98-4008-b344-21e6e839077a","Type":"ContainerStarted","Data":"d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc"}
Nov 02 03:08:12.418461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:12.418438    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:12.535076 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:12.535028    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg" podStartSLOduration=2.678451062 podStartE2EDuration="3.535013223s" totalImagesPullingTime="856.562161ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:08:12.534206648 +0000 UTC m=+1236.329555217" watchObservedRunningTime="2026-11-02 03:08:12.535013223 +0000 UTC m=+1236.330361787"
Nov 02 03:08:14.643994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:14.643958    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:14.798752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:14.798613    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:14.957885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:14.957798    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:18.319853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:18.319810    4407 patch_prober.go:29] interesting pod/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:08:18.320318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:18.319884    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:08:19.327507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.327474    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"]
Nov 02 03:08:19.335657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.335616    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.406675 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.389997    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"]
Nov 02 03:08:19.419024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.416918    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cbks\" (UniqueName: \"kubernetes.io/projected/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-kube-api-access-4cbks\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.419024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.416953    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-metrics-certs\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.419024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.417157    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.419024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.417305    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-openshift-service-ca\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.437543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.437514    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4bbf98be-serving-cert\""
Nov 02 03:08:19.438058 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.438039    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:08:19.518397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.518363    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-openshift-service-ca\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.518397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.518404    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cbks\" (UniqueName: \"kubernetes.io/projected/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-kube-api-access-4cbks\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.518658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.518436    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-metrics-certs\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.518658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.518531    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.518854 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:08:19.518833    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-serving-cert: secret "maas-api-e2e-ait-4bbf98be-serving-cert" not found
Nov 02 03:08:19.518914 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:08:19.518902    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls podName:ae9ef18a-beeb-4cf6-89c8-cea4470f220e nodeName:}" failed. No retries permitted until 2026-11-02 03:08:20.018881497 +0000 UTC m=+1243.814230042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls") pod "maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h" (UID: "ae9ef18a-beeb-4cf6-89c8-cea4470f220e") : secret "maas-api-e2e-ait-4bbf98be-serving-cert" not found
Nov 02 03:08:19.519539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.519510    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-openshift-service-ca\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.521369 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.521343    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-metrics-certs\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.544143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.544121    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:19.576821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.576796    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cbks\" (UniqueName: \"kubernetes.io/projected/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-kube-api-access-4cbks\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:19.708593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.708316    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:19.870732 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:19.870702    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:20.024693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.024617    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:20.027515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.027490    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls\") pod \"maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:20.251564 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.251531    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:20.464042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.464010    4407 generic.go:356] "Generic (PLEG): container finished" podID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerID="67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6" exitCode=0
Nov 02 03:08:20.464428 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.464085    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" event={"ID":"9dffca2b-9932-4960-b2e5-3c5f67fef0be","Type":"ContainerDied","Data":"67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6"}
Nov 02 03:08:20.464471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.464454    4407 scope.go:122] "RemoveContainer" containerID="67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6"
Nov 02 03:08:20.465746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.465716    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:20.465862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.465795    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:20.465862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.465844    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:20.708059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.707985    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"]
Nov 02 03:08:20.710740 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:08:20.710711    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9ef18a_beeb_4cf6_89c8_cea4470f220e.slice/crio-64560e02ba740fd4ff2efb89451e7ffdb98a0ebb09d06620717da4bc34fe5ae1 WatchSource:0}: Error finding container 64560e02ba740fd4ff2efb89451e7ffdb98a0ebb09d06620717da4bc34fe5ae1: Status 404 returned error can't find the container with id 64560e02ba740fd4ff2efb89451e7ffdb98a0ebb09d06620717da4bc34fe5ae1
Nov 02 03:08:20.712821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:20.712773    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 03:08:21.471034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:21.470999    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" event={"ID":"9dffca2b-9932-4960-b2e5-3c5f67fef0be","Type":"ContainerStarted","Data":"73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8"}
Nov 02 03:08:21.472287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:21.472265    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h" event={"ID":"ae9ef18a-beeb-4cf6-89c8-cea4470f220e","Type":"ContainerStarted","Data":"64560e02ba740fd4ff2efb89451e7ffdb98a0ebb09d06620717da4bc34fe5ae1"}
Nov 02 03:08:22.245130 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:22.245096    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:08:22.245307 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:22.245270    4407 patch_prober.go:29] interesting pod/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 02 03:08:22.245352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:22.245310    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 02 03:08:22.478334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:22.478293    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h" event={"ID":"ae9ef18a-beeb-4cf6-89c8-cea4470f220e","Type":"ContainerStarted","Data":"e287c008fd953743f35a81f4fe7014318e3663e8eb9b812bbe0c0a8a08a95119"}
Nov 02 03:08:22.478791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:22.478574    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:22.609708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:22.608607    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h" podStartSLOduration=2.687069683 podStartE2EDuration="3.608586901s" totalImagesPullingTime="921.517218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:08:22.607911315 +0000 UTC m=+1246.403259881" watchObservedRunningTime="2026-11-02 03:08:22.608586901 +0000 UTC m=+1246.403935480"
Nov 02 03:08:23.245069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:23.245038    4407 patch_prober.go:29] interesting pod/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 02 03:08:23.245218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:23.245099    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 02 03:08:24.245879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:24.245839    4407 patch_prober.go:29] interesting pod/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 02 03:08:24.246243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:24.245913    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 02 03:08:24.498614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:24.498524    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:25.245791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:25.245760    4407 patch_prober.go:29] interesting pod/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 02 03:08:25.245973 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:25.245812    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 02 03:08:25.877268 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:25.877234    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:08:26.244843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:26.244812    4407 patch_prober.go:29] interesting pod/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 02 03:08:26.245036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:26.244870    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 02 03:08:27.249230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:27.249194    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:08:27.250163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:27.250147    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:08:27.250687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:27.250671    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:08:28.534850 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:28.534818    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:08:28.545274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:28.545245    4407 generic.go:356] "Generic (PLEG): container finished" podID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerID="86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d" exitCode=0
Nov 02 03:08:28.545436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:28.545319    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" event={"ID":"9d554f51-8f1d-46e8-9703-f7e4e5578ac4","Type":"ContainerDied","Data":"86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d"}
Nov 02 03:08:28.545935 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:28.545915    4407 scope.go:122] "RemoveContainer" containerID="86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d"
Nov 02 03:08:28.547134 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:28.547106    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:28.547196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:28.547180    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:28.547234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:28.547217    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:29.267895 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:29.267847    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:29.565694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:29.565551    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" event={"ID":"9d554f51-8f1d-46e8-9703-f7e4e5578ac4","Type":"ContainerStarted","Data":"914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69"}
Nov 02 03:08:29.640649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:29.640156    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:08:29.640649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:29.640322    4407 patch_prober.go:29] interesting pod/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 02 03:08:29.640649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:29.640366    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 02 03:08:30.637092 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:30.637048    4407 patch_prober.go:29] interesting pod/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 02 03:08:30.638508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:30.638476    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 02 03:08:31.636098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:31.636058    4407 patch_prober.go:29] interesting pod/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 02 03:08:31.636399 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:31.636371    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 02 03:08:32.215757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:32.215724    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:32.277909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:32.277871    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:08:32.389651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:32.385497    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:32.636071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:32.636032    4407 patch_prober.go:29] interesting pod/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 02 03:08:32.636255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:32.636091    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 02 03:08:33.635199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:33.635161    4407 patch_prober.go:29] interesting pod/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 02 03:08:33.635733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:33.635213    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 02 03:08:34.169266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:34.169199    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:34.332833 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:34.332793    4407 patch_prober.go:29] interesting pod/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:08:34.333030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:34.332866    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:08:34.635480 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:34.635438    4407 patch_prober.go:29] interesting pod/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 02 03:08:34.635893 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:34.635509    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 02 03:08:35.087071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:35.087034    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:35.191342 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:08:35.191294    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:08:35.640992 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:35.640965    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:08:35.642272 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:35.642246    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:08:36.620473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.620442    4407 generic.go:356] "Generic (PLEG): container finished" podID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerID="10df0bd1694665d487af1eaccd2b7a48e3f5eebe1a5590fececd8cca0aa4ef33" exitCode=0
Nov 02 03:08:36.620748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.620509    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" event={"ID":"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03","Type":"ContainerDied","Data":"10df0bd1694665d487af1eaccd2b7a48e3f5eebe1a5590fececd8cca0aa4ef33"}
Nov 02 03:08:36.620956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.620939    4407 scope.go:122] "RemoveContainer" containerID="10df0bd1694665d487af1eaccd2b7a48e3f5eebe1a5590fececd8cca0aa4ef33"
Nov 02 03:08:36.622170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.622151    4407 generic.go:356] "Generic (PLEG): container finished" podID="303d7d7c-516b-4fee-8212-3bab14188a08" containerID="96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7" exitCode=0
Nov 02 03:08:36.622273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.622221    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" event={"ID":"303d7d7c-516b-4fee-8212-3bab14188a08","Type":"ContainerDied","Data":"96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7"}
Nov 02 03:08:36.622273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.622252    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:36.622362 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.622311    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:36.622362 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.622341    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:36.622663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.622647    4407 scope.go:122] "RemoveContainer" containerID="96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7"
Nov 02 03:08:36.623845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.623819    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:36.623916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.623892    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:36.623964 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:36.623941    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:08:37.635910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:37.635878    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" event={"ID":"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03","Type":"ContainerStarted","Data":"c0982a2c9a8c04dcdfebb1702731e4c0977e1007a88569d8a00682a9ef86a681"}
Nov 02 03:08:37.637792 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:37.637763    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" event={"ID":"303d7d7c-516b-4fee-8212-3bab14188a08","Type":"ContainerStarted","Data":"97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a"}
Nov 02 03:08:38.197554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:38.197517    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:08:38.198008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:38.197984    4407 patch_prober.go:29] interesting pod/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 02 03:08:38.198178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:38.198154    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 02 03:08:38.312963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:38.312929    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:08:38.418063 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:38.418030    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:08:38.418421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:38.418336    4407 patch_prober.go:29] interesting pod/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 02 03:08:38.418588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:38.418566    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 02 03:08:39.191098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:39.191057    4407 patch_prober.go:29] interesting pod/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 02 03:08:39.191767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:39.191731    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 02 03:08:39.416084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:39.416043    4407 patch_prober.go:29] interesting pod/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 02 03:08:39.419650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:39.416353    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 02 03:08:39.532781 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:39.532742    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:40.192652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:40.190058    4407 patch_prober.go:29] interesting pod/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 02 03:08:40.192652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:40.190124    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 02 03:08:40.418126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:40.418082    4407 patch_prober.go:29] interesting pod/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 02 03:08:40.418460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:40.418420    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 02 03:08:41.189221 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:41.189187    4407 patch_prober.go:29] interesting pod/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 02 03:08:41.189496 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:41.189468    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 02 03:08:41.419533 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:41.419083    4407 patch_prober.go:29] interesting pod/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 02 03:08:41.419533 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:41.419145    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 02 03:08:42.196650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:42.194120    4407 patch_prober.go:29] interesting pod/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 02 03:08:42.196650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:42.194187    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 02 03:08:42.421898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:42.421863    4407 patch_prober.go:29] interesting pod/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 02 03:08:42.422531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:42.422496    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 02 03:08:43.191368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:43.191338    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:08:43.192409 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:43.192387    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:08:43.419682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:43.419658    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:08:43.419951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:43.419934    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:08:43.420546 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:43.420527    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:08:44.669166 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:44.669129    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:46.502485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:46.502454    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:46.657047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:46.656996    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:49.809819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:49.809784    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:54.205654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.205603    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"]
Nov 02 03:08:54.206444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.206411    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph" podUID="0d0bea4d-8d70-4e34-898d-0af0607e7f1f" containerName="maas-api" containerID="cri-o://628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f" gracePeriod=30
Nov 02 03:08:54.344951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.344913    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:54.478761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.478734    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:08:54.602778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.602740    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-metrics-certs\") pod \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") "
Nov 02 03:08:54.602974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.602924    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-openshift-service-ca\") pod \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") "
Nov 02 03:08:54.602974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.602967    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls\") pod \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") "
Nov 02 03:08:54.603108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.603002    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-kube-api-access-brcgr\" (UniqueName: \"kubernetes.io/projected/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-kube-api-access-brcgr\") pod \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\" (UID: \"0d0bea4d-8d70-4e34-898d-0af0607e7f1f\") "
Nov 02 03:08:54.603490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.603452    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-openshift-service-ca" pod "0d0bea4d-8d70-4e34-898d-0af0607e7f1f" (UID: "0d0bea4d-8d70-4e34-898d-0af0607e7f1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:08:54.606438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.606406    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls" pod "0d0bea4d-8d70-4e34-898d-0af0607e7f1f" (UID: "0d0bea4d-8d70-4e34-898d-0af0607e7f1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:08:54.606438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.606415    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-kube-api-access-brcgr" pod "0d0bea4d-8d70-4e34-898d-0af0607e7f1f" (UID: "0d0bea4d-8d70-4e34-898d-0af0607e7f1f"). InnerVolumeSpecName "kube-api-access-brcgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:08:54.620054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.618869    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-metrics-certs" pod "0d0bea4d-8d70-4e34-898d-0af0607e7f1f" (UID: "0d0bea4d-8d70-4e34-898d-0af0607e7f1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:08:54.704754 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.704723    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:08:54.704754 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.704747    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:08:54.704754 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.704756    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:08:54.704754 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.704764    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brcgr\" (UniqueName: \"kubernetes.io/projected/0d0bea4d-8d70-4e34-898d-0af0607e7f1f-kube-api-access-brcgr\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:08:54.725197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.725100    4407 generic.go:356] "Generic (PLEG): container finished" podID="0d0bea4d-8d70-4e34-898d-0af0607e7f1f" containerID="628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f" exitCode=0
Nov 02 03:08:54.725197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.725142    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"
Nov 02 03:08:54.725197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.725173    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph" event={"ID":"0d0bea4d-8d70-4e34-898d-0af0607e7f1f","Type":"ContainerDied","Data":"628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f"}
Nov 02 03:08:54.725465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.725234    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph" event={"ID":"0d0bea4d-8d70-4e34-898d-0af0607e7f1f","Type":"ContainerDied","Data":"7ad766758f76d285f0abb39ecf3c9340eada516a84e954b472c3838c502f5ee5"}
Nov 02 03:08:54.725465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.725262    4407 scope.go:122] "RemoveContainer" containerID="628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f"
Nov 02 03:08:54.735323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.735306    4407 scope.go:122] "RemoveContainer" containerID="628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f"
Nov 02 03:08:54.735596 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:08:54.735577    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f\": container with ID starting with 628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f not found: ID does not exist" containerID="628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f"
Nov 02 03:08:54.735706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.735606    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f"} err="failed to get container status \"628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f\": rpc error: code = NotFound desc = could not find container \"628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f\": container with ID starting with 628cf24b190a23b60480f089b50855c992b7190ce5b899d2de9dd750697c601f not found: ID does not exist"
Nov 02 03:08:54.889751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.889719    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"]
Nov 02 03:08:54.891344 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:54.891324    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d1107301-7fd646f6df-5sfph"]
Nov 02 03:08:55.548702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:55.548672    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:55.983855 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:55.983768    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:56.146355 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:56.146323    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:56.297570 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:56.297543    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:56.350514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:56.350480    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d0bea4d-8d70-4e34-898d-0af0607e7f1f" path="/var/lib/kubelet/pods/0d0bea4d-8d70-4e34-898d-0af0607e7f1f/volumes"
Nov 02 03:08:56.454164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:56.454130    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:56.715067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:56.714960    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:57.441556 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:57.441508    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:57.596568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:57.596537    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:58.017216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:58.017181    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:58.325048 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:58.324970    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:59.108837 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:59.108802    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:59.399792 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:59.399710    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:08:59.783852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:59.783814    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"]
Nov 02 03:08:59.784110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:59.784067    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy" containerID="cri-o://73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8" gracePeriod=30
Nov 02 03:08:59.806506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:08:59.806470    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:00.471051 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:00.471000    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:00.932672 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:00.932632    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"]
Nov 02 03:09:00.933087 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:00.933074    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0d0bea4d-8d70-4e34-898d-0af0607e7f1f" containerName="maas-api"
Nov 02 03:09:00.933143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:00.933088    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d0bea4d-8d70-4e34-898d-0af0607e7f1f" containerName="maas-api"
Nov 02 03:09:00.933187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:00.933176    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d0bea4d-8d70-4e34-898d-0af0607e7f1f" containerName="maas-api"
Nov 02 03:09:00.936852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:00.936835    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.033773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.033742    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"]
Nov 02 03:09:01.038876 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.038858    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-858236-openshift-default-dockercfg-79wxp\""
Nov 02 03:09:01.064551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.064523    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:01.077742 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.077688    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.078074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.077748    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.078074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.077810    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a35abc-96d2-419e-b183-b8eba22cb573-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.078074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.077835    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a35abc-96d2-419e-b183-b8eba22cb573-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.078074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.077874    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5msld\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-kube-api-access-5msld\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.078074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.077919    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.078074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.077969    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.078074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.078058    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.078531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.078091    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.086253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.086216    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:09:01.178983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.178952    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179006    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179026    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179062    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a35abc-96d2-419e-b183-b8eba22cb573-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179087    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a35abc-96d2-419e-b183-b8eba22cb573-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179112    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5msld\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-kube-api-access-5msld\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179144    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179208    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179309    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179424    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179513    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179649    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179713    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.179938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.179914    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a35abc-96d2-419e-b183-b8eba22cb573-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.182196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.182171    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.182286 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.182269    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a35abc-96d2-419e-b183-b8eba22cb573-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.236891 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.236816    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:01.237554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.237517    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.237712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.237645    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5msld\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-kube-api-access-5msld\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.249417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.249393    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:01.474128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.474100    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:01.634275 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.634249    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:01.679814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.679790    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"]
Nov 02 03:09:01.680824 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:09:01.680798    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a35abc_96d2_419e_b183_b8eba22cb573.slice/crio-1a4a4cf5c9227321c04f631e24ee2fc76fcc04a82703e560493bef76c215c2b0 WatchSource:0}: Error finding container 1a4a4cf5c9227321c04f631e24ee2fc76fcc04a82703e560493bef76c215c2b0: Status 404 returned error can't find the container with id 1a4a4cf5c9227321c04f631e24ee2fc76fcc04a82703e560493bef76c215c2b0
Nov 02 03:09:01.682995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.682961    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:01.683063 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.683028    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:01.683063 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.683058    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:01.762192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.762158    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" event={"ID":"55a35abc-96d2-419e-b183-b8eba22cb573","Type":"ContainerStarted","Data":"258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9"}
Nov 02 03:09:01.762376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.762228    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" event={"ID":"55a35abc-96d2-419e-b183-b8eba22cb573","Type":"ContainerStarted","Data":"1a4a4cf5c9227321c04f631e24ee2fc76fcc04a82703e560493bef76c215c2b0"}
Nov 02 03:09:01.879816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.879767    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" podStartSLOduration=1.879754006 podStartE2EDuration="1.879754006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:09:01.878864632 +0000 UTC m=+1285.674213209" watchObservedRunningTime="2026-11-02 03:09:01.879754006 +0000 UTC m=+1285.675102570"
Nov 02 03:09:01.916122 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.916040    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"]
Nov 02 03:09:01.916302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:01.916276    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg" podUID="1fdf10b4-8a98-4008-b344-21e6e839077a" containerName="maas-api" containerID="cri-o://d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc" gracePeriod=30
Nov 02 03:09:02.043036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.043008    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:02.168323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.168260    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:09:02.249965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.249931    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:02.254952 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.254928    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:02.290174 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.290145    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fdf10b4-8a98-4008-b344-21e6e839077a-kube-api-access-qth2b\" (UniqueName: \"kubernetes.io/projected/1fdf10b4-8a98-4008-b344-21e6e839077a-kube-api-access-qth2b\") pod \"1fdf10b4-8a98-4008-b344-21e6e839077a\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") "
Nov 02 03:09:02.290319 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.290250    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-metrics-certs\") pod \"1fdf10b4-8a98-4008-b344-21e6e839077a\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") "
Nov 02 03:09:02.290319 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.290305    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls\") pod \"1fdf10b4-8a98-4008-b344-21e6e839077a\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") "
Nov 02 03:09:02.290440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.290348    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fdf10b4-8a98-4008-b344-21e6e839077a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fdf10b4-8a98-4008-b344-21e6e839077a-openshift-service-ca\") pod \"1fdf10b4-8a98-4008-b344-21e6e839077a\" (UID: \"1fdf10b4-8a98-4008-b344-21e6e839077a\") "
Nov 02 03:09:02.291042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.291009    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fdf10b4-8a98-4008-b344-21e6e839077a-openshift-service-ca" pod "1fdf10b4-8a98-4008-b344-21e6e839077a" (UID: "1fdf10b4-8a98-4008-b344-21e6e839077a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:09:02.296982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.296941    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-metrics-certs" pod "1fdf10b4-8a98-4008-b344-21e6e839077a" (UID: "1fdf10b4-8a98-4008-b344-21e6e839077a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:09:02.297097 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.297009    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls" pod "1fdf10b4-8a98-4008-b344-21e6e839077a" (UID: "1fdf10b4-8a98-4008-b344-21e6e839077a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:09:02.297097 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.297045    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fdf10b4-8a98-4008-b344-21e6e839077a-kube-api-access-qth2b" pod "1fdf10b4-8a98-4008-b344-21e6e839077a" (UID: "1fdf10b4-8a98-4008-b344-21e6e839077a"). InnerVolumeSpecName "kube-api-access-qth2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:02.391794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.391765    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:02.391794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.391789    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fdf10b4-8a98-4008-b344-21e6e839077a-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:02.391794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.391799    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fdf10b4-8a98-4008-b344-21e6e839077a-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:02.391993 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.391807    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qth2b\" (UniqueName: \"kubernetes.io/projected/1fdf10b4-8a98-4008-b344-21e6e839077a-kube-api-access-qth2b\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:02.571426 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.571386    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:02.765799 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.765768    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:02.768912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.768881    4407 generic.go:356] "Generic (PLEG): container finished" podID="1fdf10b4-8a98-4008-b344-21e6e839077a" containerID="d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc" exitCode=0
Nov 02 03:09:02.769056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.768928    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"
Nov 02 03:09:02.769056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.768975    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg" event={"ID":"1fdf10b4-8a98-4008-b344-21e6e839077a","Type":"ContainerDied","Data":"d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc"}
Nov 02 03:09:02.769056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.769020    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg" event={"ID":"1fdf10b4-8a98-4008-b344-21e6e839077a","Type":"ContainerDied","Data":"89f0d840d6c8ef372437c710e079beda869e507aae3a3c03028c58fb569e0845"}
Nov 02 03:09:02.769056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.769041    4407 scope.go:122] "RemoveContainer" containerID="d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc"
Nov 02 03:09:02.769764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.769319    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:02.770812 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.770463    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:02.780485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.780468    4407 scope.go:122] "RemoveContainer" containerID="d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc"
Nov 02 03:09:02.780907 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:02.780874    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc\": container with ID starting with d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc not found: ID does not exist" containerID="d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc"
Nov 02 03:09:02.781027 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.780924    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc"} err="failed to get container status \"d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc\": rpc error: code = NotFound desc = could not find container \"d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc\": container with ID starting with d77ea8b80e9014d5eb8bc23925a8f73a3c02c1b6e3bef7135f028a86f56428bc not found: ID does not exist"
Nov 02 03:09:02.931899 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.931790    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:02.944768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.944740    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"]
Nov 02 03:09:02.952751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:02.949106    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-66a47a3b-645cc89754-wqbbg"]
Nov 02 03:09:03.957815 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:03.957783    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:04.350125 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:04.350094    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fdf10b4-8a98-4008-b344-21e6e839077a" path="/var/lib/kubelet/pods/1fdf10b4-8a98-4008-b344-21e6e839077a/volumes"
Nov 02 03:09:05.046985 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.046922    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:09:05.188424 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:05.188389    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:09:05.220033 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.219998    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-data\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220064    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-socket\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220108    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-credential-socket\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220145    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-token\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-token\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220190    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-kube-api-access-x6r57\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-kube-api-access-x6r57\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220249    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-podinfo\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220295    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-certs\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220312    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-data" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:05.220459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220387    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-envoy\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220457    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istiod-ca-cert\") pod \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\" (UID: \"9dffca2b-9932-4960-b2e5-3c5f67fef0be\") "
Nov 02 03:09:05.220795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220597    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-credential-socket" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:05.220890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220869    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.220947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220896    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.221005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220954    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-socket" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:05.221005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.220969    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-certs" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:05.221110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.221012    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istiod-ca-cert" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:09:05.222988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.222961    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-token" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:05.223132 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.223106    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-kube-api-access-x6r57" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "kube-api-access-x6r57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:05.223218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.223201    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-envoy" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:05.223306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.223287    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-podinfo" pod "9dffca2b-9932-4960-b2e5-3c5f67fef0be" (UID: "9dffca2b-9932-4960-b2e5-3c5f67fef0be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:09:05.322407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.322327    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.322407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.322357    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.322407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.322367    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6r57\" (UniqueName: \"kubernetes.io/projected/9dffca2b-9932-4960-b2e5-3c5f67fef0be-kube-api-access-x6r57\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.322407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.322377    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.322407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.322386    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.322407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.322394    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.322407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.322402    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dffca2b-9932-4960-b2e5-3c5f67fef0be-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:05.785951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.785916    4407 generic.go:356] "Generic (PLEG): container finished" podID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerID="73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8" exitCode=0
Nov 02 03:09:05.786140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.785959    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"
Nov 02 03:09:05.786140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.786007    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" event={"ID":"9dffca2b-9932-4960-b2e5-3c5f67fef0be","Type":"ContainerDied","Data":"73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8"}
Nov 02 03:09:05.786140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.786063    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b" event={"ID":"9dffca2b-9932-4960-b2e5-3c5f67fef0be","Type":"ContainerDied","Data":"44bc5f6d360f11fbb9b0b2e78822527bbae249f57432aaa234da28bf90b76781"}
Nov 02 03:09:05.786140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.786087    4407 scope.go:122] "RemoveContainer" containerID="73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8"
Nov 02 03:09:05.795447 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.795428    4407 scope.go:122] "RemoveContainer" containerID="67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6"
Nov 02 03:09:05.803381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.803365    4407 scope.go:122] "RemoveContainer" containerID="73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8"
Nov 02 03:09:05.803608 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:05.803590    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8\": container with ID starting with 73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8 not found: ID does not exist" containerID="73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8"
Nov 02 03:09:05.803700 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.803636    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8"} err="failed to get container status \"73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8\": rpc error: code = NotFound desc = could not find container \"73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8\": container with ID starting with 73f3133f73799bcd5b616895fa6bf725ac44a6851c37f6e44b28667ea22f2df8 not found: ID does not exist"
Nov 02 03:09:05.803700 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.803662    4407 scope.go:122] "RemoveContainer" containerID="67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6"
Nov 02 03:09:05.803932 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:05.803915    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6\": container with ID starting with 67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6 not found: ID does not exist" containerID="67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6"
Nov 02 03:09:05.803967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:05.803940    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6"} err="failed to get container status \"67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6\": rpc error: code = NotFound desc = could not find container \"67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6\": container with ID starting with 67af58d56ea4540d5e9e09cac03823d8edf1daedd9933905f19d00c9b2cf0ca6 not found: ID does not exist"
Nov 02 03:09:06.002447 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:06.002411    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"]
Nov 02 03:09:06.053039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:06.052967    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-935e4abf-gw-openshift-default-759dd556b5-gz55b"]
Nov 02 03:09:06.241980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:06.241948    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:06.350651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:06.350560    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" path="/var/lib/kubelet/pods/9dffca2b-9932-4960-b2e5-3c5f67fef0be/volumes"
Nov 02 03:09:06.541638 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:06.541598    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:06.692568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:06.692491    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:06.845926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:06.845898    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:07.181369 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:07.181337    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:07.357646 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:07.357597    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:07.525973 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:07.525941    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:08.141410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:08.141374    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"]
Nov 02 03:09:08.141654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:08.141611    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy" containerID="cri-o://914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69" gracePeriod=30
Nov 02 03:09:08.158859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:08.158825    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:08.884488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:08.884446    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:09.133916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.133877    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:09.374519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.374483    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"]
Nov 02 03:09:09.374920 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.374908    4407 cpu_manager.go:414] "containerMap: removing container" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy"
Nov 02 03:09:09.374982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.374921    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy"
Nov 02 03:09:09.374982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.374936    4407 cpu_manager.go:414] "containerMap: removing container" podUID="1fdf10b4-8a98-4008-b344-21e6e839077a" containerName="maas-api"
Nov 02 03:09:09.374982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.374941    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fdf10b4-8a98-4008-b344-21e6e839077a" containerName="maas-api"
Nov 02 03:09:09.374982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.374947    4407 cpu_manager.go:414] "containerMap: removing container" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy"
Nov 02 03:09:09.374982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.374953    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy"
Nov 02 03:09:09.375128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.375026    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy"
Nov 02 03:09:09.375128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.375034    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fdf10b4-8a98-4008-b344-21e6e839077a" containerName="maas-api"
Nov 02 03:09:09.375128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.375039    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dffca2b-9932-4960-b2e5-3c5f67fef0be" containerName="istio-proxy"
Nov 02 03:09:09.379607 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.379588    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.433060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.433025    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-398828-openshift-default-dockercfg-dqq8t\""
Nov 02 03:09:09.466516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466480    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-certs\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.466708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466557    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-podinfo\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.466708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466656    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7086420f-22a5-4dff-b91c-fa81c4d4e279-istiod-ca-cert\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.466708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466695    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-token\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.466815 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466763    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-data\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.466815 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466799    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.466887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466849    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-credential-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.466887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466869    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4rkz\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-kube-api-access-t4rkz\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.466990 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.466972    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-envoy\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.485087 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.485061    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"]
Nov 02 03:09:09.498355 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.498330    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:09.568095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568062    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7086420f-22a5-4dff-b91c-fa81c4d4e279-istiod-ca-cert\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568111    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-token\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568140    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-data\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568159    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568189    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-credential-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568208    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4rkz\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-kube-api-access-t4rkz\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568250    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-envoy\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568301    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-certs\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568348    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-podinfo\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568753 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568664    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568753 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568675    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-credential-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568762    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-certs\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568863    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-data\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.568930 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.568912    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7086420f-22a5-4dff-b91c-fa81c4d4e279-istiod-ca-cert\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.570765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.570748    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-envoy\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.570879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.570863    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-podinfo\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.626661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.626567    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4rkz\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-kube-api-access-t4rkz\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.627011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.626993    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-token\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-ph22f\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.691994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.691970    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:09.769810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:09.769778    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:10.121862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.121834    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"]
Nov 02 03:09:10.123612 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:09:10.123584    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7086420f_22a5_4dff_b91c_fa81c4d4e279.slice/crio-7a0415672f7b5051de9ddee89abe76f3bf97da74909d559df2d49b82796d49c7 WatchSource:0}: Error finding container 7a0415672f7b5051de9ddee89abe76f3bf97da74909d559df2d49b82796d49c7: Status 404 returned error can't find the container with id 7a0415672f7b5051de9ddee89abe76f3bf97da74909d559df2d49b82796d49c7
Nov 02 03:09:10.126657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.126590    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:10.126724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.126680    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:10.126724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.126713    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:10.435169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.435091    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:10.580253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.580226    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:10.813972 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.813935    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" event={"ID":"7086420f-22a5-4dff-b91c-fa81c4d4e279","Type":"ContainerStarted","Data":"17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3"}
Nov 02 03:09:10.814147 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.813987    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" event={"ID":"7086420f-22a5-4dff-b91c-fa81c4d4e279","Type":"ContainerStarted","Data":"7a0415672f7b5051de9ddee89abe76f3bf97da74909d559df2d49b82796d49c7"}
Nov 02 03:09:10.933024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:10.932964    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" podStartSLOduration=1.932950019 podStartE2EDuration="1.932950019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:09:10.930757247 +0000 UTC m=+1294.726105812" watchObservedRunningTime="2026-11-02 03:09:10.932950019 +0000 UTC m=+1294.728298630"
Nov 02 03:09:11.692514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:11.692479    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:11.697726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:11.697699    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:11.820342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:11.820314    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:11.821287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:11.821264    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:12.863528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:12.863486    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:13.023536 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.023500    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:13.397548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.397526    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:09:13.406791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.406727    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istiod-ca-cert\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.406791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.406776    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-envoy\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.406965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.406819    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-token\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-token\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.406965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.406867    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-data\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.406965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.406923    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-kube-api-access-vwb24\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-kube-api-access-vwb24\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.406965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.406944    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-podinfo\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.407207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.407034    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-credential-socket\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.407207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.407066    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-socket\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.407207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.407098    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istiod-ca-cert" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:09:13.407207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.407112    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-certs\") pod \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\" (UID: \"9d554f51-8f1d-46e8-9703-f7e4e5578ac4\") "
Nov 02 03:09:13.407520 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.407407    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-credential-socket" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:13.408096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.407957    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-certs" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:13.408096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.408046    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-data" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:13.408096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.408061    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-socket" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:13.408334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.408231    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.408334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.408253    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.409320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.409291    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-token" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:13.409431 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.409416    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-podinfo" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:09:13.409605 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.409574    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-envoy" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:13.410210 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.410190    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-kube-api-access-vwb24" pod "9d554f51-8f1d-46e8-9703-f7e4e5578ac4" (UID: "9d554f51-8f1d-46e8-9703-f7e4e5578ac4"). InnerVolumeSpecName "kube-api-access-vwb24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:13.509112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.509079    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.509112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.509107    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.509112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.509116    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.509352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.509126    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwb24\" (UniqueName: \"kubernetes.io/projected/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-kube-api-access-vwb24\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.509352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.509136    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.509352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.509145    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.509352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.509153    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d554f51-8f1d-46e8-9703-f7e4e5578ac4-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:13.673459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.673375    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:13.831258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.831227    4407 generic.go:356] "Generic (PLEG): container finished" podID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerID="914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69" exitCode=0
Nov 02 03:09:13.831421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.831277    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"
Nov 02 03:09:13.831421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.831308    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" event={"ID":"9d554f51-8f1d-46e8-9703-f7e4e5578ac4","Type":"ContainerDied","Data":"914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69"}
Nov 02 03:09:13.831421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.831360    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k" event={"ID":"9d554f51-8f1d-46e8-9703-f7e4e5578ac4","Type":"ContainerDied","Data":"0297db6024a162e4958fb9657456487230cab1a95e76500e00ef5746c74f6ac8"}
Nov 02 03:09:13.831421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.831381    4407 scope.go:122] "RemoveContainer" containerID="914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69"
Nov 02 03:09:13.840637 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.840598    4407 scope.go:122] "RemoveContainer" containerID="86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d"
Nov 02 03:09:13.849061 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.849037    4407 scope.go:122] "RemoveContainer" containerID="914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69"
Nov 02 03:09:13.849319 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:13.849302    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69\": container with ID starting with 914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69 not found: ID does not exist" containerID="914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69"
Nov 02 03:09:13.849379 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.849330    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69"} err="failed to get container status \"914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69\": rpc error: code = NotFound desc = could not find container \"914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69\": container with ID starting with 914cf760d9f63febf31887a7113b8dd3ed11e75f881c5af7ca8dc0d983632d69 not found: ID does not exist"
Nov 02 03:09:13.849379 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.849350    4407 scope.go:122] "RemoveContainer" containerID="86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d"
Nov 02 03:09:13.849606 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:13.849579    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d\": container with ID starting with 86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d not found: ID does not exist" containerID="86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d"
Nov 02 03:09:13.849689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.849636    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d"} err="failed to get container status \"86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d\": rpc error: code = NotFound desc = could not find container \"86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d\": container with ID starting with 86ef07b21eb11031ae554ae780ed214905c48fb76301576b7a569a62dd7e008d not found: ID does not exist"
Nov 02 03:09:13.865118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.865086    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:13.997329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:13.997299    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"]
Nov 02 03:09:14.037537 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:14.037502    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:14.047874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:14.047836    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-66a47a3b-openshift-default-5fdf96d89c-vzn7k"]
Nov 02 03:09:14.314542 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:14.314463    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:14.352571 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:14.352542    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" path="/var/lib/kubelet/pods/9d554f51-8f1d-46e8-9703-f7e4e5578ac4/volumes"
Nov 02 03:09:16.297575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:16.297543    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:16.479052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:16.479020    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:17.221633 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:17.221599    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:09:18.916012 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:18.915936    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:19.071598 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.071569    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:19.312514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.312483    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"]
Nov 02 03:09:19.312946 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.312933    4407 cpu_manager.go:414] "containerMap: removing container" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy"
Nov 02 03:09:19.313011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.312948    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy"
Nov 02 03:09:19.313011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.312958    4407 cpu_manager.go:414] "containerMap: removing container" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy"
Nov 02 03:09:19.313011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.312963    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy"
Nov 02 03:09:19.313119 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.313109    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy"
Nov 02 03:09:19.313150 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.313119    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d554f51-8f1d-46e8-9703-f7e4e5578ac4" containerName="istio-proxy"
Nov 02 03:09:19.317992 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.317912    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.368611 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.368568    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.368785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.368648    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-metrics-certs\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.368785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.368746    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dwm8\" (UniqueName: \"kubernetes.io/projected/325388df-f4cd-4944-a901-39e1da3558cb-kube-api-access-5dwm8\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.368882 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.368874    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/325388df-f4cd-4944-a901-39e1da3558cb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.426310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.426278    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"]
Nov 02 03:09:19.430982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.430957    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-ca6546-serving-cert\""
Nov 02 03:09:19.470162 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.470113    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.470338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.470191    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-metrics-certs\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.470338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.470239    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dwm8\" (UniqueName: \"kubernetes.io/projected/325388df-f4cd-4944-a901-39e1da3558cb-kube-api-access-5dwm8\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.470338 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:19.470273    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-serving-cert: secret "maas-api-e2e-worker-w2-ca6546-serving-cert" not found
Nov 02 03:09:19.470528 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:19.470348    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls podName:325388df-f4cd-4944-a901-39e1da3558cb nodeName:}" failed. No retries permitted until 2026-11-02 03:09:19.970325416 +0000 UTC m=+1303.765673981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls") pod "maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t" (UID: "325388df-f4cd-4944-a901-39e1da3558cb") : secret "maas-api-e2e-worker-w2-ca6546-serving-cert" not found
Nov 02 03:09:19.470528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.470417    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/325388df-f4cd-4944-a901-39e1da3558cb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.471188 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.471164    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/325388df-f4cd-4944-a901-39e1da3558cb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.473311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.473282    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-metrics-certs\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.530193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.530162    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dwm8\" (UniqueName: \"kubernetes.io/projected/325388df-f4cd-4944-a901-39e1da3558cb-kube-api-access-5dwm8\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.558611 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.558572    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:19.741578 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.741504    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:19.976518 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.976485    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:19.979208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:19.979184    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:20.238777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:20.238746    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:20.609153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:20.609130    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"]
Nov 02 03:09:20.611037 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:09:20.611007    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325388df_f4cd_4944_a901_39e1da3558cb.slice/crio-5a7b5dc82fa5034b358682acefefa65b46e34002f916990238083d7c4235be8f WatchSource:0}: Error finding container 5a7b5dc82fa5034b358682acefefa65b46e34002f916990238083d7c4235be8f: Status 404 returned error can't find the container with id 5a7b5dc82fa5034b358682acefefa65b46e34002f916990238083d7c4235be8f
Nov 02 03:09:20.868604 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:20.868514    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t" event={"ID":"325388df-f4cd-4944-a901-39e1da3558cb","Type":"ContainerStarted","Data":"5a7b5dc82fa5034b358682acefefa65b46e34002f916990238083d7c4235be8f"}
Nov 02 03:09:21.875322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:21.875233    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t" event={"ID":"325388df-f4cd-4944-a901-39e1da3558cb","Type":"ContainerStarted","Data":"e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2"}
Nov 02 03:09:21.875322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:21.875301    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:21.998276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:21.998229    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t" podStartSLOduration=2.112097785 podStartE2EDuration="2.998216199s" totalImagesPullingTime="886.118414ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:09:21.997929483 +0000 UTC m=+1305.793278048" watchObservedRunningTime="2026-11-02 03:09:21.998216199 +0000 UTC m=+1305.793564765"
Nov 02 03:09:24.406576 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:24.406541    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:24.571601 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:24.571570    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:24.725347 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:24.725270    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:27.885912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:27.885885    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 03:09:28.779297 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:28.779267    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:29.748345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:29.748312    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"]
Nov 02 03:09:29.748831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:29.748531    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h" podUID="ae9ef18a-beeb-4cf6-89c8-cea4470f220e" containerName="maas-api" containerID="cri-o://e287c008fd953743f35a81f4fe7014318e3663e8eb9b812bbe0c0a8a08a95119" gracePeriod=30
Nov 02 03:09:29.872149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:29.872120    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:29.916799 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:29.916765    4407 generic.go:356] "Generic (PLEG): container finished" podID="ae9ef18a-beeb-4cf6-89c8-cea4470f220e" containerID="e287c008fd953743f35a81f4fe7014318e3663e8eb9b812bbe0c0a8a08a95119" exitCode=0
Nov 02 03:09:29.916961 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:29.916834    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h" event={"ID":"ae9ef18a-beeb-4cf6-89c8-cea4470f220e","Type":"ContainerDied","Data":"e287c008fd953743f35a81f4fe7014318e3663e8eb9b812bbe0c0a8a08a95119"}
Nov 02 03:09:30.030541 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.030513    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:09:30.182398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.182366    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-kube-api-access-4cbks\" (UniqueName: \"kubernetes.io/projected/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-kube-api-access-4cbks\") pod \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") "
Nov 02 03:09:30.182565 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.182409    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-metrics-certs\") pod \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") "
Nov 02 03:09:30.182565 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.182549    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-openshift-service-ca\") pod \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") "
Nov 02 03:09:30.182689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.182575    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls\") pod \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\" (UID: \"ae9ef18a-beeb-4cf6-89c8-cea4470f220e\") "
Nov 02 03:09:30.182983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.182962    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-openshift-service-ca" pod "ae9ef18a-beeb-4cf6-89c8-cea4470f220e" (UID: "ae9ef18a-beeb-4cf6-89c8-cea4470f220e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:09:30.184738 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.184707    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-metrics-certs" pod "ae9ef18a-beeb-4cf6-89c8-cea4470f220e" (UID: "ae9ef18a-beeb-4cf6-89c8-cea4470f220e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:09:30.184892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.184766    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-kube-api-access-4cbks" pod "ae9ef18a-beeb-4cf6-89c8-cea4470f220e" (UID: "ae9ef18a-beeb-4cf6-89c8-cea4470f220e"). InnerVolumeSpecName "kube-api-access-4cbks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:30.184892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.184825    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls" pod "ae9ef18a-beeb-4cf6-89c8-cea4470f220e" (UID: "ae9ef18a-beeb-4cf6-89c8-cea4470f220e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:09:30.236095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.236058    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:30.283905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.283828    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:30.283905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.283857    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:30.283905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.283870    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cbks\" (UniqueName: \"kubernetes.io/projected/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-kube-api-access-4cbks\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:30.283905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.283880    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae9ef18a-beeb-4cf6-89c8-cea4470f220e-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:30.519813 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.519780    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:30.804994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.804961    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"]
Nov 02 03:09:30.805372 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.805210    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy" containerID="cri-o://c0982a2c9a8c04dcdfebb1702731e4c0977e1007a88569d8a00682a9ef86a681" gracePeriod=30
Nov 02 03:09:30.823146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.823115    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:30.923374 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.923346    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"
Nov 02 03:09:30.923538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.923341    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h" event={"ID":"ae9ef18a-beeb-4cf6-89c8-cea4470f220e","Type":"ContainerDied","Data":"64560e02ba740fd4ff2efb89451e7ffdb98a0ebb09d06620717da4bc34fe5ae1"}
Nov 02 03:09:30.923538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:30.923488    4407 scope.go:122] "RemoveContainer" containerID="e287c008fd953743f35a81f4fe7014318e3663e8eb9b812bbe0c0a8a08a95119"
Nov 02 03:09:31.089886 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.089790    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"]
Nov 02 03:09:31.093397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.093368    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4bbf98be-698c48df84-gxm5h"]
Nov 02 03:09:31.431656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.431564    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:31.533537 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.533506    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"]
Nov 02 03:09:31.534034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.534020    4407 cpu_manager.go:414] "containerMap: removing container" podUID="ae9ef18a-beeb-4cf6-89c8-cea4470f220e" containerName="maas-api"
Nov 02 03:09:31.534086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.534036    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae9ef18a-beeb-4cf6-89c8-cea4470f220e" containerName="maas-api"
Nov 02 03:09:31.534118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.534106    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae9ef18a-beeb-4cf6-89c8-cea4470f220e" containerName="maas-api"
Nov 02 03:09:31.538481 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.538465    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.592597 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.592568    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-398828-serving-cert\""
Nov 02 03:09:31.599653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.599614    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"]
Nov 02 03:09:31.684741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.684636    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:31.699597 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.699564    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f404c24-4402-4b99-8e03-a3818d6032ec-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.699773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.699756    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.699829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.699786    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcmsw\" (UniqueName: \"kubernetes.io/projected/1f404c24-4402-4b99-8e03-a3818d6032ec-kube-api-access-xcmsw\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.699829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.699817    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-metrics-certs\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.800730 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.800689    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.800924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.800735    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcmsw\" (UniqueName: \"kubernetes.io/projected/1f404c24-4402-4b99-8e03-a3818d6032ec-kube-api-access-xcmsw\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.800924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.800781    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-metrics-certs\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.800924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.800814    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f404c24-4402-4b99-8e03-a3818d6032ec-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.800924 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:31.800840    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-serving-cert: secret "maas-api-e2e-worker-w4-398828-serving-cert" not found
Nov 02 03:09:31.800924 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:31.800904    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls podName:1f404c24-4402-4b99-8e03-a3818d6032ec nodeName:}" failed. No retries permitted until 2026-11-02 03:09:32.300885072 +0000 UTC m=+1316.096233615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls") pod "maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx" (UID: "1f404c24-4402-4b99-8e03-a3818d6032ec") : secret "maas-api-e2e-worker-w4-398828-serving-cert" not found
Nov 02 03:09:31.801494 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.801474    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f404c24-4402-4b99-8e03-a3818d6032ec-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.803542 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.803519    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-metrics-certs\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:31.852524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.852496    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:31.864491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:31.864459    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcmsw\" (UniqueName: \"kubernetes.io/projected/1f404c24-4402-4b99-8e03-a3818d6032ec-kube-api-access-xcmsw\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:32.011506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.011472    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:32.185333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.185304    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:09:32.194875 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.194848    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:32.306504 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.306417    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:32.308991 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.308971    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls\") pod \"maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:32.350787 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.350757    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae9ef18a-beeb-4cf6-89c8-cea4470f220e" path="/var/lib/kubelet/pods/ae9ef18a-beeb-4cf6-89c8-cea4470f220e/volumes"
Nov 02 03:09:32.451061 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.451021    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:32.506399 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.506367    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:32.507445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.507422    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"]
Nov 02 03:09:32.507687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.507663    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy" containerID="cri-o://97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a" gracePeriod=30
Nov 02 03:09:32.673153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.673068    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:32.855735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.855713    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"]
Nov 02 03:09:32.857473 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:09:32.857442    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f404c24_4402_4b99_8e03_a3818d6032ec.slice/crio-d7d03c37809055c239bda1dadadfc4535514406e0d9d04dbaded4d1785a2ae64 WatchSource:0}: Error finding container d7d03c37809055c239bda1dadadfc4535514406e0d9d04dbaded4d1785a2ae64: Status 404 returned error can't find the container with id d7d03c37809055c239bda1dadadfc4535514406e0d9d04dbaded4d1785a2ae64
Nov 02 03:09:32.934928 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:32.934848    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx" event={"ID":"1f404c24-4402-4b99-8e03-a3818d6032ec","Type":"ContainerStarted","Data":"d7d03c37809055c239bda1dadadfc4535514406e0d9d04dbaded4d1785a2ae64"}
Nov 02 03:09:33.304658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.304057    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:33.769664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.769616    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"]
Nov 02 03:09:33.780432 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.777213    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.833704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.833638    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"]
Nov 02 03:09:33.871661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.871616    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-1f6e39-openshift-default-dockercfg-qc8xh\""
Nov 02 03:09:33.905777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.905400    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:33.923115 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923069    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.923290 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923195    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.923345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923293    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29tcd\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-kube-api-access-29tcd\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.923345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923334    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.923410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923368    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.923410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923396    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.923545 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923520    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.923650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923606    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.923715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.923695    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:33.942064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.942032    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx" event={"ID":"1f404c24-4402-4b99-8e03-a3818d6032ec","Type":"ContainerStarted","Data":"391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7"}
Nov 02 03:09:33.942222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:33.942147    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:34.024482 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024440    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.024728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024521    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29tcd\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-kube-api-access-29tcd\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.024728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024558    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.024728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024601    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.024728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024674    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.024728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024720    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.025005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024773    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.025005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024903    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.025005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024926    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.025005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.024956    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.025222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.025151    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.025329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.025308    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.025411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.025394    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.025654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.025582    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.027224 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.027198    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.027441 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.027423    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.067864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.067808    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx" podStartSLOduration=2.198874627 podStartE2EDuration="3.067792772s" totalImagesPullingTime="868.918145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:09:34.0675363 +0000 UTC m=+1317.862884865" watchObservedRunningTime="2026-11-02 03:09:34.067792772 +0000 UTC m=+1317.863141344"
Nov 02 03:09:34.101647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.101421    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29tcd\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-kube-api-access-29tcd\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.106112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.106080    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.124544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.124499    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:34.303887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.301430    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:34.552915 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.552893    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"]
Nov 02 03:09:34.554294 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:09:34.554240    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b57248_092f_4876_b4f2_ce7bfb44fe7b.slice/crio-6de1cb43a8b92c94f8560c0ee4441ed706d49509e7358709033a35eb913e83e8 WatchSource:0}: Error finding container 6de1cb43a8b92c94f8560c0ee4441ed706d49509e7358709033a35eb913e83e8: Status 404 returned error can't find the container with id 6de1cb43a8b92c94f8560c0ee4441ed706d49509e7358709033a35eb913e83e8
Nov 02 03:09:34.556662 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.556615    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:34.556726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.556699    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:34.556768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.556727    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:34.950172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.950070    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" event={"ID":"a3b57248-092f-4876-b4f2-ce7bfb44fe7b","Type":"ContainerStarted","Data":"b189826f864ccecb8abfdc8434084f12d388caada9077b7dec3e9ee6683778a3"}
Nov 02 03:09:34.950172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:34.950141    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" event={"ID":"a3b57248-092f-4876-b4f2-ce7bfb44fe7b","Type":"ContainerStarted","Data":"6de1cb43a8b92c94f8560c0ee4441ed706d49509e7358709033a35eb913e83e8"}
Nov 02 03:09:35.069605 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.069538    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podStartSLOduration=2.069521052 podStartE2EDuration="2.069521052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:09:35.068804286 +0000 UTC m=+1318.864152851" watchObservedRunningTime="2026-11-02 03:09:35.069521052 +0000 UTC m=+1318.864869618"
Nov 02 03:09:35.125114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.125074    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:35.130138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.130114    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:35.219849 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:35.219767    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:09:35.446860 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.446801    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:35.958527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.958488    4407 generic.go:356] "Generic (PLEG): container finished" podID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerID="c0982a2c9a8c04dcdfebb1702731e4c0977e1007a88569d8a00682a9ef86a681" exitCode=0
Nov 02 03:09:35.959133 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.959098    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" event={"ID":"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03","Type":"ContainerDied","Data":"c0982a2c9a8c04dcdfebb1702731e4c0977e1007a88569d8a00682a9ef86a681"}
Nov 02 03:09:35.959263 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.959162    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:35.959263 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.959189    4407 scope.go:122] "RemoveContainer" containerID="10df0bd1694665d487af1eaccd2b7a48e3f5eebe1a5590fececd8cca0aa4ef33"
Nov 02 03:09:35.960802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:35.960786    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:09:36.104909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.104824    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:09:36.249807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.249778    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-podinfo\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.249971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.249879    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-data\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.249971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.249948    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istiod-ca-cert\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.250060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.249977    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-credential-socket\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.250060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250013    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-token\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-token\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.250138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250060    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-kube-api-access-d8f8v\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-kube-api-access-d8f8v\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.250138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250107    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-certs\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.250431 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250330    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-data" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:36.250431 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250436    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-envoy\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.250786 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250495    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-socket\") pod \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\" (UID: \"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03\") "
Nov 02 03:09:36.250786 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250668    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istiod-ca-cert" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:09:36.250786 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250662    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-credential-socket" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:36.250955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250893    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-socket" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:36.251018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.250936    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-certs" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:36.251137 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.251107    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.251201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.251140    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.251201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.251149    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.251201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.251158    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.251201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.251166    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.252250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.252226    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-podinfo" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:09:36.253139 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.253115    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-kube-api-access-d8f8v" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "kube-api-access-d8f8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:36.253261 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.253222    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-token" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:36.253914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.253896    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-envoy" pod "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" (UID: "bfe5b198-f1b9-4d32-8e34-6faa1c22fc03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:36.351603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.351577    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.351603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.351602    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.351804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.351611    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8f8v\" (UniqueName: \"kubernetes.io/projected/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-kube-api-access-d8f8v\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.351804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.351650    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:36.965739 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.965710    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"
Nov 02 03:09:36.965739 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.965712    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt" event={"ID":"bfe5b198-f1b9-4d32-8e34-6faa1c22fc03","Type":"ContainerDied","Data":"1b48f8c035b785f09b1fea5c89c2569c4ec19672fc1a58eb61df304c5c3b03c4"}
Nov 02 03:09:36.966253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:36.965783    4407 scope.go:122] "RemoveContainer" containerID="c0982a2c9a8c04dcdfebb1702731e4c0977e1007a88569d8a00682a9ef86a681"
Nov 02 03:09:37.189658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.182896    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"]
Nov 02 03:09:37.232780 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.232701    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-bc460382-openshift-default-565fc557b5-lbtjt"]
Nov 02 03:09:37.340883 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.340853    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:37.572426 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.572397    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:37.760751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.760728    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:37.770504 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.770454    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:09:37.867490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867454    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-data\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.867685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867535    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/303d7d7c-516b-4fee-8212-3bab14188a08-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/303d7d7c-516b-4fee-8212-3bab14188a08-istio-podinfo\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.867685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867570    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-socket\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.867685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867645    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-istio-token\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-istio-token\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.867843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867746    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-certs\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.867843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867745    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-data" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:37.867843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867792    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/303d7d7c-516b-4fee-8212-3bab14188a08-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/303d7d7c-516b-4fee-8212-3bab14188a08-istiod-ca-cert\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.867843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867825    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-envoy\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.868055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867856    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-kube-api-access-vcph5\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-kube-api-access-vcph5\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.868055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867894    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-credential-socket\") pod \"303d7d7c-516b-4fee-8212-3bab14188a08\" (UID: \"303d7d7c-516b-4fee-8212-3bab14188a08\") "
Nov 02 03:09:37.868055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.867901    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-socket" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:37.868055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.868000    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-certs" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:37.868263 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.868124    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/303d7d7c-516b-4fee-8212-3bab14188a08-istiod-ca-cert" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:09:37.868263 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.868235    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-credential-socket" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:37.868263 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.868256    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.868548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.868278    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/303d7d7c-516b-4fee-8212-3bab14188a08-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.868548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.868293    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.868548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.868308    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.870057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.870029    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/303d7d7c-516b-4fee-8212-3bab14188a08-istio-podinfo" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:09:37.870169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.870118    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-istio-token" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:37.870232 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.870217    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-envoy" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:09:37.870467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.870454    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-kube-api-access-vcph5" pod "303d7d7c-516b-4fee-8212-3bab14188a08" (UID: "303d7d7c-516b-4fee-8212-3bab14188a08"). InnerVolumeSpecName "kube-api-access-vcph5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:09:37.922500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.922458    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:37.969295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.969269    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.969295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.969294    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcph5\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-kube-api-access-vcph5\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.969295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.969304    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/303d7d7c-516b-4fee-8212-3bab14188a08-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.969780 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.969313    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/303d7d7c-516b-4fee-8212-3bab14188a08-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.969780 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.969322    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/303d7d7c-516b-4fee-8212-3bab14188a08-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:09:37.973499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.973471    4407 generic.go:356] "Generic (PLEG): container finished" podID="303d7d7c-516b-4fee-8212-3bab14188a08" containerID="97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a" exitCode=0
Nov 02 03:09:37.973645 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.973519    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"
Nov 02 03:09:37.973645 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.973552    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" event={"ID":"303d7d7c-516b-4fee-8212-3bab14188a08","Type":"ContainerDied","Data":"97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a"}
Nov 02 03:09:37.973750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.973646    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr" event={"ID":"303d7d7c-516b-4fee-8212-3bab14188a08","Type":"ContainerDied","Data":"94a2d452e880b8b6ce764532f20e0b3b4d5e7f121fe4a1a83e8b82c7efd07673"}
Nov 02 03:09:37.973750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:37.973674    4407 scope.go:122] "RemoveContainer" containerID="97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a"
Nov 02 03:09:38.009814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.009780    4407 scope.go:122] "RemoveContainer" containerID="96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7"
Nov 02 03:09:38.019593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.019571    4407 scope.go:122] "RemoveContainer" containerID="97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a"
Nov 02 03:09:38.019938 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:38.019918    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a\": container with ID starting with 97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a not found: ID does not exist" containerID="97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a"
Nov 02 03:09:38.020014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.019952    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a"} err="failed to get container status \"97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a\": rpc error: code = NotFound desc = could not find container \"97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a\": container with ID starting with 97d76a865d248cec56fe71451e0ecb3b53553dcb604bfc57fdbc11984c09223a not found: ID does not exist"
Nov 02 03:09:38.020014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.019970    4407 scope.go:122] "RemoveContainer" containerID="96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7"
Nov 02 03:09:38.020230 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:09:38.020212    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7\": container with ID starting with 96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7 not found: ID does not exist" containerID="96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7"
Nov 02 03:09:38.020275 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.020240    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7"} err="failed to get container status \"96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7\": rpc error: code = NotFound desc = could not find container \"96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7\": container with ID starting with 96f4f4fdfd7662884393ea1f1175853c9fd5dbf166b3be68c142632a80b062e7 not found: ID does not exist"
Nov 02 03:09:38.159928 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.159849    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"]
Nov 02 03:09:38.169841 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.169816    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4bbf98be-openshift-default-5446cd585c-g4hdr"]
Nov 02 03:09:38.236815 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.236787    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:38.350637 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.350598    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" path="/var/lib/kubelet/pods/303d7d7c-516b-4fee-8212-3bab14188a08/volumes"
Nov 02 03:09:38.351111 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.351097    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" path="/var/lib/kubelet/pods/bfe5b198-f1b9-4d32-8e34-6faa1c22fc03/volumes"
Nov 02 03:09:38.423215 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.423132    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:38.607225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:38.607192    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:39.955753 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:39.955723    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 03:09:40.805240 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:40.805210    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:41.002173 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:41.002141    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:41.459630 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:41.459590    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:09:42.926021 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:42.925988    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:43.021674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:43.021643    4407 generic.go:356] "Generic (PLEG): container finished" podID="55a35abc-96d2-419e-b183-b8eba22cb573" containerID="258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9" exitCode=0
Nov 02 03:09:43.021847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:43.021694    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" event={"ID":"55a35abc-96d2-419e-b183-b8eba22cb573","Type":"ContainerDied","Data":"258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9"}
Nov 02 03:09:43.022094 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:43.022074    4407 scope.go:122] "RemoveContainer" containerID="258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9"
Nov 02 03:09:43.023363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:43.023337    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:43.023434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:43.023402    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:43.023474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:43.023432    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:44.035333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:44.035281    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" event={"ID":"55a35abc-96d2-419e-b183-b8eba22cb573","Type":"ContainerStarted","Data":"dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545"}
Nov 02 03:09:44.249947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:44.249914    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:44.250999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:44.250978    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 02 03:09:44.251082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:44.251025    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 02 03:09:45.250188 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:45.250157    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 02 03:09:45.250572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:45.250206    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 02 03:09:45.624717 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:45.621715    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:45.831335 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:45.831180    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:46.250205 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:46.250172    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 02 03:09:46.250649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:46.250228    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 02 03:09:47.250223 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:47.250191    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 02 03:09:47.250656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:47.250244    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 02 03:09:47.769640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:47.769578    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:48.110879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:48.110850    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:48.249859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:48.249825    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 02 03:09:48.250001 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:48.249874    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 02 03:09:49.330816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:49.330496    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:49.332640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:49.332590    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 03:09:52.079284 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:52.079250    4407 generic.go:356] "Generic (PLEG): container finished" podID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerID="17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3" exitCode=0
Nov 02 03:09:52.079750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:52.079328    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" event={"ID":"7086420f-22a5-4dff-b91c-fa81c4d4e279","Type":"ContainerDied","Data":"17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3"}
Nov 02 03:09:52.079750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:52.079731    4407 scope.go:122] "RemoveContainer" containerID="17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3"
Nov 02 03:09:52.080992 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:52.080961    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:52.081098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:52.081045    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:52.081161 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:52.081095    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:09:53.026773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:53.026705    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:53.109590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:53.109554    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" event={"ID":"7086420f-22a5-4dff-b91c-fa81c4d4e279","Type":"ContainerStarted","Data":"5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d"}
Nov 02 03:09:53.372535 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:53.372446    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:53.601536 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:53.601465    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:53.696707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:53.692965    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:53.705652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:53.703014    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-ph22f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 02 03:09:53.705652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:53.703081    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 02 03:09:53.778150 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:53.778115    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:54.698464 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:54.698032    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-ph22f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 02 03:09:54.698464 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:54.698093    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 02 03:09:55.198649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:55.197341    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:55.700650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:55.695016    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-ph22f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 02 03:09:55.700650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:55.695090    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 02 03:09:55.720001 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:55.719975    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:55.898663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:55.898121    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:56.705651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:56.703728    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-ph22f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 02 03:09:56.705651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:56.703786    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 02 03:09:56.821117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:56.821084    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:57.573448 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:57.573416    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:57.703649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:57.703100    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-ph22f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 02 03:09:57.703649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:57.703212    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 02 03:09:58.183556 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.183516    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"]
Nov 02 03:09:58.184182 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184163    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy"
Nov 02 03:09:58.184225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184188    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy"
Nov 02 03:09:58.184259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184228    4407 cpu_manager.go:414] "containerMap: removing container" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy"
Nov 02 03:09:58.184259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184238    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy"
Nov 02 03:09:58.184259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184250    4407 cpu_manager.go:414] "containerMap: removing container" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy"
Nov 02 03:09:58.184259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184258    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy"
Nov 02 03:09:58.184387 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184375    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy"
Nov 02 03:09:58.184422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184397    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy"
Nov 02 03:09:58.184422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.184409    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="303d7d7c-516b-4fee-8212-3bab14188a08" containerName="istio-proxy"
Nov 02 03:09:58.188052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.188034    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.291851 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.291817    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-5ac93f-serving-cert\""
Nov 02 03:09:58.293955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.293927    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"]
Nov 02 03:09:58.339528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.339494    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dba853-b2ce-4244-aa40-e1cea64d5af0-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.339774 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.339726    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-metrics-certs\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.339912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.339890    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x27ns\" (UniqueName: \"kubernetes.io/projected/14dba853-b2ce-4244-aa40-e1cea64d5af0-kube-api-access-x27ns\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.339963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.339936    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-maas-api-tls\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.394987 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.394947    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:58.440734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.440649    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-metrics-certs\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.440734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.440710    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x27ns\" (UniqueName: \"kubernetes.io/projected/14dba853-b2ce-4244-aa40-e1cea64d5af0-kube-api-access-x27ns\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.440734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.440736    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-maas-api-tls\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.440983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.440760    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dba853-b2ce-4244-aa40-e1cea64d5af0-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.441372 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.441352    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dba853-b2ce-4244-aa40-e1cea64d5af0-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.443234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.443210    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-maas-api-tls\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.443322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.443248    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-metrics-certs\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.497899 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.497869    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x27ns\" (UniqueName: \"kubernetes.io/projected/14dba853-b2ce-4244-aa40-e1cea64d5af0-kube-api-access-x27ns\") pod \"maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.500717 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.500698    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:09:58.548495 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.548436    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:58.697467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.697395    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:58.698510 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.698485    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 03:09:58.699413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.699392    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:09:58.873367 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:58.873339    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"]
Nov 02 03:09:58.874874 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:09:58.874854    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14dba853_b2ce_4244_aa40_e1cea64d5af0.slice/crio-1df94052c0dad023807000cd4e27f49fcf432749196fdac66792778d4c59e152 WatchSource:0}: Error finding container 1df94052c0dad023807000cd4e27f49fcf432749196fdac66792778d4c59e152: Status 404 returned error can't find the container with id 1df94052c0dad023807000cd4e27f49fcf432749196fdac66792778d4c59e152
Nov 02 03:09:59.148338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:09:59.148305    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5" event={"ID":"14dba853-b2ce-4244-aa40-e1cea64d5af0","Type":"ContainerStarted","Data":"1df94052c0dad023807000cd4e27f49fcf432749196fdac66792778d4c59e152"}
Nov 02 03:10:00.155033 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:00.154939    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5" event={"ID":"14dba853-b2ce-4244-aa40-e1cea64d5af0","Type":"ContainerStarted","Data":"8e94b85297acf5187caa2691c2e2d37e53a2a83c39449e6e0ece7662b3ebaf96"}
Nov 02 03:10:00.155463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:00.155039    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:10:00.272230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:00.272177    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5" podStartSLOduration=1.403211412 podStartE2EDuration="2.272162333s" totalImagesPullingTime="868.950921ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:10:00.270188778 +0000 UTC m=+1344.065537342" watchObservedRunningTime="2026-11-02 03:10:00.272162333 +0000 UTC m=+1344.067510896"
Nov 02 03:10:03.295834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.295804    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"]
Nov 02 03:10:03.296249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.296236    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy"
Nov 02 03:10:03.296310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.296250    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy"
Nov 02 03:10:03.296345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.296332    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfe5b198-f1b9-4d32-8e34-6faa1c22fc03" containerName="istio-proxy"
Nov 02 03:10:03.299460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.299444    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.353603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.353559    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-1f6e39-serving-cert\""
Nov 02 03:10:03.388324 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.388283    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grhns\" (UniqueName: \"kubernetes.io/projected/90cc541b-2f24-4f28-8045-99e7736a1b7f-kube-api-access-grhns\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.388501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.388427    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.388501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.388453    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90cc541b-2f24-4f28-8045-99e7736a1b7f-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.388592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.388570    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-metrics-certs\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.406699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.406645    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"]
Nov 02 03:10:03.489797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.489766    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-metrics-certs\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.489942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.489836    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grhns\" (UniqueName: \"kubernetes.io/projected/90cc541b-2f24-4f28-8045-99e7736a1b7f-kube-api-access-grhns\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.489942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.489892    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.489942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.489909    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90cc541b-2f24-4f28-8045-99e7736a1b7f-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.490065 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:10:03.489988    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-serving-cert: secret "maas-api-e2e-worker-w0-1f6e39-serving-cert" not found
Nov 02 03:10:03.490108 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:10:03.490065    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls podName:90cc541b-2f24-4f28-8045-99e7736a1b7f nodeName:}" failed. No retries permitted until 2026-11-02 03:10:03.990042026 +0000 UTC m=+1347.785390573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls") pod "maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265" (UID: "90cc541b-2f24-4f28-8045-99e7736a1b7f") : secret "maas-api-e2e-worker-w0-1f6e39-serving-cert" not found
Nov 02 03:10:03.490531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.490515    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90cc541b-2f24-4f28-8045-99e7736a1b7f-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.492299 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.492281    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-metrics-certs\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.518853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.518823    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:03.550921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.550843    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grhns\" (UniqueName: \"kubernetes.io/projected/90cc541b-2f24-4f28-8045-99e7736a1b7f-kube-api-access-grhns\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.676669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.676605    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:03.831778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.831698    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:03.995051 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.995020    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:03.997551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:03.997524    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls\") pod \"maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:04.061371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:04.061338    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:04.212394 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:04.212316    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:04.239139 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:04.238825    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:10:04.635360 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:04.635338    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"]
Nov 02 03:10:04.637015 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:10:04.636986    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90cc541b_2f24_4f28_8045_99e7736a1b7f.slice/crio-2680393414acf69bd3f025cc24b4f9ff77d3beac920cb25a359a72ff99891d8b WatchSource:0}: Error finding container 2680393414acf69bd3f025cc24b4f9ff77d3beac920cb25a359a72ff99891d8b: Status 404 returned error can't find the container with id 2680393414acf69bd3f025cc24b4f9ff77d3beac920cb25a359a72ff99891d8b
Nov 02 03:10:04.639731 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:04.639706    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:04.820315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:04.820287    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:05.180411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:05.180379    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265" event={"ID":"90cc541b-2f24-4f28-8045-99e7736a1b7f","Type":"ContainerStarted","Data":"2680393414acf69bd3f025cc24b4f9ff77d3beac920cb25a359a72ff99891d8b"}
Nov 02 03:10:05.188561 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:10:05.188532    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:10:06.165449 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:06.165424    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 03:10:06.186777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:06.186747    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265" event={"ID":"90cc541b-2f24-4f28-8045-99e7736a1b7f","Type":"ContainerStarted","Data":"91e20546e94f1a148f9d61f00d1fe2cc78f2dabcb62ee5b2c73bace4f1f06ee5"}
Nov 02 03:10:06.186924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:06.186811    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:06.398318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:06.398259    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265" podStartSLOduration=2.547716318 podStartE2EDuration="3.398240329s" totalImagesPullingTime="850.524011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:10:06.396468951 +0000 UTC m=+1350.191817518" watchObservedRunningTime="2026-11-02 03:10:06.398240329 +0000 UTC m=+1350.193588895"
Nov 02 03:10:08.658265 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:08.658213    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:12.196202 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:12.196170    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 03:10:12.634166 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:12.634134    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:10:13.470847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:13.470815    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:18.313120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:18.313085    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:22.000767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:22.000732    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:10:22.001201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:22.000795    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:10:22.263931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:22.263846    4407 generic.go:356] "Generic (PLEG): container finished" podID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerID="b189826f864ccecb8abfdc8434084f12d388caada9077b7dec3e9ee6683778a3" exitCode=0
Nov 02 03:10:22.264108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:22.263926    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" event={"ID":"a3b57248-092f-4876-b4f2-ce7bfb44fe7b","Type":"ContainerDied","Data":"b189826f864ccecb8abfdc8434084f12d388caada9077b7dec3e9ee6683778a3"}
Nov 02 03:10:22.264318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:22.264303    4407 scope.go:122] "RemoveContainer" containerID="b189826f864ccecb8abfdc8434084f12d388caada9077b7dec3e9ee6683778a3"
Nov 02 03:10:22.265561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:22.265536    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:10:22.265640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:22.265609    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:10:22.265680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:22.265659    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:10:23.271216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:23.271177    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" event={"ID":"a3b57248-092f-4876-b4f2-ce7bfb44fe7b","Type":"ContainerStarted","Data":"99b1a06056f8aaa2cb64ad48cf880bdb63dc6f0b7e354258677a15436da3fa77"}
Nov 02 03:10:24.124843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:24.124813    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:10:24.125054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:24.125024    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Nov 02 03:10:24.125126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:24.125089    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Nov 02 03:10:24.681096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:24.681061    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:24.842422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:24.842392    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:25.124938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:25.124900    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Nov 02 03:10:25.125144 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:25.124964    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Nov 02 03:10:26.125873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:26.125835    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Nov 02 03:10:26.126384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:26.125893    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Nov 02 03:10:27.125090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:27.125052    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Nov 02 03:10:27.125272 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:27.125107    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Nov 02 03:10:27.908602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:27.908570    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:28.125337 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:28.125305    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Nov 02 03:10:28.125503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:28.125356    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Nov 02 03:10:28.937756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:28.937723    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:10:29.016371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:29.016338    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:29.129159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:29.129127    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:10:29.129436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:29.129421    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:10:29.130062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:29.130044    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:10:32.774895 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:32.774862    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:35.188996 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:10:35.188959    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:10:35.779954 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:35.779925    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:37.581912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:37.581881    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:38.079796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:38.079769    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:38.237567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:38.237501    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:38.399046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:38.398950    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:42.237017 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:42.236981    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:10:42.767366 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:42.767337    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:43.178557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:43.178482    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:43.617410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:43.617374    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:44.191429 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:44.191390    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:44.359242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:44.359214    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:47.977519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:47.977435    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:52.771905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:52.771868    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:10:55.499104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:55.499065    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:10:57.571926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:10:57.571877    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:02.368368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:02.368334    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:05.188248 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:11:05.188215    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:11:06.003858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.003828    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"]
Nov 02 03:11:06.007443 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.007424    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.061838 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.061776    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-5ac93f\"/\"openshift-service-ca.crt\""
Nov 02 03:11:06.112656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.112608    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-5ac93f\"/\"kube-root-ca.crt\""
Nov 02 03:11:06.112819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.112702    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5ac93f\"/\"simulator-w1-5ac93f-kserve-self-signed-certs\""
Nov 02 03:11:06.112819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.112711    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5ac93f\"/\"default-dockercfg-4jt6g\""
Nov 02 03:11:06.114938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.114917    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"]
Nov 02 03:11:06.149455 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.149429    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-model-cache\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.149609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.149479    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccjvr\" (UniqueName: \"kubernetes.io/projected/f44a93a8-19e2-47e1-969f-c156f5c97772-kube-api-access-ccjvr\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.149609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.149580    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-home\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.149780 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.149727    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-dshm\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.149835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.149795    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-tmp-dir\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.149890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.149868    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44a93a8-19e2-47e1-969f-c156f5c97772-tls-certs\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.156382 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.156357    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:06.251036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251003    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-dshm\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.251463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251067    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-tmp-dir\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.251463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251112    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44a93a8-19e2-47e1-969f-c156f5c97772-tls-certs\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.251463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251254    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-model-cache\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.251463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251332    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccjvr\" (UniqueName: \"kubernetes.io/projected/f44a93a8-19e2-47e1-969f-c156f5c97772-kube-api-access-ccjvr\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.251463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251370    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-home\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.251726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251494    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-tmp-dir\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.251726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251575    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-model-cache\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.251977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.251955    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-home\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.253439 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.253414    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-dshm\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.253696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.253677    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44a93a8-19e2-47e1-969f-c156f5c97772-tls-certs\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.316684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.316554    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccjvr\" (UniqueName: \"kubernetes.io/projected/f44a93a8-19e2-47e1-969f-c156f5c97772-kube-api-access-ccjvr\") pod \"simulator-w1-5ac93f-kserve-57cb964469-xsv4s\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") " pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.318616 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.318599    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:06.689759 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:06.689731    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"]
Nov 02 03:11:06.691392 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:11:06.691363    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf44a93a8_19e2_47e1_969f_c156f5c97772.slice/crio-97b354e485c351428d557c102955478446b9085ffb15cdbf0bf77f4ec85016f6 WatchSource:0}: Error finding container 97b354e485c351428d557c102955478446b9085ffb15cdbf0bf77f4ec85016f6: Status 404 returned error can't find the container with id 97b354e485c351428d557c102955478446b9085ffb15cdbf0bf77f4ec85016f6
Nov 02 03:11:07.153255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:07.153211    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:07.507090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:07.507061    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" event={"ID":"f44a93a8-19e2-47e1-969f-c156f5c97772","Type":"ContainerStarted","Data":"38485bfaf6833208b3f552d2d8f3a32c7b8a4ce0e87e4bee5c45e9cc38007c74"}
Nov 02 03:11:07.507473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:07.507099    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" event={"ID":"f44a93a8-19e2-47e1-969f-c156f5c97772","Type":"ContainerStarted","Data":"97b354e485c351428d557c102955478446b9085ffb15cdbf0bf77f4ec85016f6"}
Nov 02 03:11:07.622228 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:07.622174    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podStartSLOduration=2.622160076 podStartE2EDuration="2.622160076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:11:07.621458067 +0000 UTC m=+1411.416806633" watchObservedRunningTime="2026-11-02 03:11:07.622160076 +0000 UTC m=+1411.417508640"
Nov 02 03:11:08.319023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:08.318992    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:11.966978 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:11.966940    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:13.184170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:13.184138    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:11:16.319456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:16.319419    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:16.331824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:16.331795    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:16.559688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:16.559662    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:11:16.718044 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:16.717947    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:16.875886 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:16.875848    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:21.474320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:21.474285    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:25.949007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:25.948975    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:26.280096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:26.280063    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:31.072667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:31.072614    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:32.810434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:32.810399    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:11:35.188548 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:11:35.188509    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:11:35.983796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:35.983752    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:36.145923 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:36.145886    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:40.764887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:40.764853    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:43.102650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.102605    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"]
Nov 02 03:11:43.106871 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.106850    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.193212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.193169    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-398828\"/\"openshift-service-ca.crt\""
Nov 02 03:11:43.195328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.195305    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"]
Nov 02 03:11:43.211509 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.211486    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-398828\"/\"kube-root-ca.crt\""
Nov 02 03:11:43.211616 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.211605    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-398828\"/\"default-dockercfg-n7lmt\""
Nov 02 03:11:43.211699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.211677    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-398828\"/\"premium-w4-398828-kserve-self-signed-certs\""
Nov 02 03:11:43.215767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.215749    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:43.218971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.218954    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-model-cache\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.219019 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.218986    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tls-certs\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.219061 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.219025    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwgqd\" (UniqueName: \"kubernetes.io/projected/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-kube-api-access-vwgqd\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.219140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.219130    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tmp-dir\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.219186 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.219176    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-home\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.219223 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.219192    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-dshm\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320404 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320372    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-home\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320404 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320406    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-dshm\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320447    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-model-cache\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320480    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tls-certs\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320537    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwgqd\" (UniqueName: \"kubernetes.io/projected/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-kube-api-access-vwgqd\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320596    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tmp-dir\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320856    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-home\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320875    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-model-cache\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.320980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.320950    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tmp-dir\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.322963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.322943    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-dshm\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.323131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.323115    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tls-certs\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.381403 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.381319    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwgqd\" (UniqueName: \"kubernetes.io/projected/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-kube-api-access-vwgqd\") pod \"premium-w4-398828-kserve-574875d9d4-wrm5t\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") " pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.418234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.418201    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:43.788378 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:43.788351    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"]
Nov 02 03:11:43.790042 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:11:43.790012    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec2f3b6e_98d1_42bc_8718_a15b56f874a9.slice/crio-f5a478464840ca071a20934a11bb7647c6c18da4df10c836611276bf7805e859 WatchSource:0}: Error finding container f5a478464840ca071a20934a11bb7647c6c18da4df10c836611276bf7805e859: Status 404 returned error can't find the container with id f5a478464840ca071a20934a11bb7647c6c18da4df10c836611276bf7805e859
Nov 02 03:11:44.691013 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:44.690976    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" event={"ID":"ec2f3b6e-98d1-42bc-8718-a15b56f874a9","Type":"ContainerStarted","Data":"8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d"}
Nov 02 03:11:44.691412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:44.691022    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" event={"ID":"ec2f3b6e-98d1-42bc-8718-a15b56f874a9","Type":"ContainerStarted","Data":"f5a478464840ca071a20934a11bb7647c6c18da4df10c836611276bf7805e859"}
Nov 02 03:11:44.807830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:44.807781    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podStartSLOduration=1.8077677859999999 podStartE2EDuration="1.807767786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:11:44.806403002 +0000 UTC m=+1448.601751570" watchObservedRunningTime="2026-11-02 03:11:44.807767786 +0000 UTC m=+1448.603116351"
Nov 02 03:11:45.418748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:45.418718    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:45.571265 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:45.571222    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:50.076840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:50.076806    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:11:50.367058 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:50.366982    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:52.743109 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:52.743077    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:53.419319 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:53.419290    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:53.431973 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:53.431948    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:53.750525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:53.750488    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:11:55.107663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:55.107603    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:11:59.871753 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:11:59.871718    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:04.694372 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:04.694338    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:05.187584 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:12:05.187550    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:12:08.027229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:08.027191    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:12:09.608050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:09.608020    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:12.797938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.797890    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"]
Nov 02 03:12:12.801979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.801958    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:12.858681 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.858646    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"]
Nov 02 03:12:12.906302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.906280    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5ac93f\"/\"premium-w1-5ac93f-kserve-self-signed-certs\""
Nov 02 03:12:12.909293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.909272    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:12.933725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.933693    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4q5s\" (UniqueName: \"kubernetes.io/projected/173c0706-37a9-4e23-823d-ffbf6e67be49-kube-api-access-h4q5s\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:12.933852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.933736    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-model-cache\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:12.933852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.933762    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-dshm\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:12.933960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.933853    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-tmp-dir\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:12.933960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.933876    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/173c0706-37a9-4e23-823d-ffbf6e67be49-tls-certs\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:12.934043 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:12.934024    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-home\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.034922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.034892    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-home\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.035099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.034981    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4q5s\" (UniqueName: \"kubernetes.io/projected/173c0706-37a9-4e23-823d-ffbf6e67be49-kube-api-access-h4q5s\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.035099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.035001    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-model-cache\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.035099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.035017    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-dshm\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.035099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.035037    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-tmp-dir\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.035099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.035054    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/173c0706-37a9-4e23-823d-ffbf6e67be49-tls-certs\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.035353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.035303    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-home\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.035459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.035438    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-model-cache\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.035515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.035471    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-tmp-dir\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.037422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.037402    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-dshm\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.037551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.037534    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/173c0706-37a9-4e23-823d-ffbf6e67be49-tls-certs\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.092518 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.092456    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4q5s\" (UniqueName: \"kubernetes.io/projected/173c0706-37a9-4e23-823d-ffbf6e67be49-kube-api-access-h4q5s\") pod \"premium-w1-5ac93f-kserve-74bf649df6-gjwrf\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") " pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.114458 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.114436    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:13.485192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.485166    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"]
Nov 02 03:12:13.486919 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:12:13.486879    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod173c0706_37a9_4e23_823d_ffbf6e67be49.slice/crio-780faa6bd2fca7b3791657e9c43013f8337ea56daf220edd53b83803369091c1 WatchSource:0}: Error finding container 780faa6bd2fca7b3791657e9c43013f8337ea56daf220edd53b83803369091c1: Status 404 returned error can't find the container with id 780faa6bd2fca7b3791657e9c43013f8337ea56daf220edd53b83803369091c1
Nov 02 03:12:13.842557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.842524    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" event={"ID":"173c0706-37a9-4e23-823d-ffbf6e67be49","Type":"ContainerStarted","Data":"c66590ffe9c18b056e7045184e89ea00f73f399e0b5db0f56f7f27ccf05dbc9b"}
Nov 02 03:12:13.842557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.842566    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" event={"ID":"173c0706-37a9-4e23-823d-ffbf6e67be49","Type":"ContainerStarted","Data":"780faa6bd2fca7b3791657e9c43013f8337ea56daf220edd53b83803369091c1"}
Nov 02 03:12:13.961801 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:13.961745    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podStartSLOduration=1.961731922 podStartE2EDuration="1.961731922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:12:13.960061957 +0000 UTC m=+1477.755410521" watchObservedRunningTime="2026-11-02 03:12:13.961731922 +0000 UTC m=+1477.757080487"
Nov 02 03:12:14.115319 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:14.115235    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:14.477996 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:14.477920    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:14.912476 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:14.912448    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:19.196684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:19.196587    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:23.115384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:23.115347    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:23.128021 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:23.127991    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:23.901920 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:23.901894    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:12:24.032372 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:24.032335    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:28.784963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:28.784928    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:33.415652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.415605    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"]
Nov 02 03:12:33.419355 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.419332    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.452531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.452499    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c3e132-eb14-4eed-ad62-b7018519081b-tls-certs\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.452729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.452568    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpzls\" (UniqueName: \"kubernetes.io/projected/71c3e132-eb14-4eed-ad62-b7018519081b-kube-api-access-mpzls\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.452729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.452652    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-dshm\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.452729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.452696    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-model-cache\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.452885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.452811    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-home\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.452955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.452933    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-tmp-dir\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.519213 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.519180    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:33.523983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.523958    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-1f6e39\"/\"openshift-service-ca.crt\""
Nov 02 03:12:33.523983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.523971    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1f6e39\"/\"default-dockercfg-tb4qg\""
Nov 02 03:12:33.524180 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.524012    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-1f6e39\"/\"kube-root-ca.crt\""
Nov 02 03:12:33.524180 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.524054    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1f6e39\"/\"premium-w0-1f6e39-kserve-self-signed-certs\""
Nov 02 03:12:33.525956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.525938    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"]
Nov 02 03:12:33.554149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554121    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-dshm\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.554311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554167    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-model-cache\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.554311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554262    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-home\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.554412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554311    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-tmp-dir\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.554412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554343    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c3e132-eb14-4eed-ad62-b7018519081b-tls-certs\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.554412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554392    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpzls\" (UniqueName: \"kubernetes.io/projected/71c3e132-eb14-4eed-ad62-b7018519081b-kube-api-access-mpzls\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.554713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554682    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-model-cache\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.554835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554727    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-home\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.554835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.554771    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-tmp-dir\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.556602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.556579    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-dshm\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.556908 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.556890    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c3e132-eb14-4eed-ad62-b7018519081b-tls-certs\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.612328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.612298    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpzls\" (UniqueName: \"kubernetes.io/projected/71c3e132-eb14-4eed-ad62-b7018519081b-kube-api-access-mpzls\") pod \"premium-w0-1f6e39-kserve-85768b88b8-zrdz5\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") " pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:33.678519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.678439    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:33.732182 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:33.732155    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:34.109736 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:34.109713    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"]
Nov 02 03:12:34.111320 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:12:34.111294    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71c3e132_eb14_4eed_ad62_b7018519081b.slice/crio-870eafbd63cd816b596a4b132ed46e812ae46a77ec522ec65dc499a89386b619 WatchSource:0}: Error finding container 870eafbd63cd816b596a4b132ed46e812ae46a77ec522ec65dc499a89386b619: Status 404 returned error can't find the container with id 870eafbd63cd816b596a4b132ed46e812ae46a77ec522ec65dc499a89386b619
Nov 02 03:12:34.946267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:34.946226    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" event={"ID":"71c3e132-eb14-4eed-ad62-b7018519081b","Type":"ContainerStarted","Data":"2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b"}
Nov 02 03:12:34.946267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:34.946272    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" event={"ID":"71c3e132-eb14-4eed-ad62-b7018519081b","Type":"ContainerStarted","Data":"870eafbd63cd816b596a4b132ed46e812ae46a77ec522ec65dc499a89386b619"}
Nov 02 03:12:34.987682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:34.987649    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:12:35.062424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:35.062371    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podStartSLOduration=2.062357433 podStartE2EDuration="2.062357433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:12:35.060917068 +0000 UTC m=+1498.856265634" watchObservedRunningTime="2026-11-02 03:12:35.062357433 +0000 UTC m=+1498.857705997"
Nov 02 03:12:35.191458 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:12:35.191417    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:12:35.732693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:35.732667    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:36.471045 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:36.471013    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:38.419509 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:38.419472    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:40.978283 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:40.978251    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:41.151957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:41.151929    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:43.264198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:43.264166    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:43.732950 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:43.732861    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:43.748252 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:43.748221    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:44.003737 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:44.003712    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:12:48.077498 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:48.077463    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:52.882828 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:52.882794    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:12:57.674678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:12:57.674648    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:02.468963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:02.468931    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:03.863571 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:03.863538    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:13:05.190771 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:13:05.190733    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:13:07.361148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:07.361121    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:12.220115 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:12.220081    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:13.915459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:13.915427    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:14.075874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:14.075839    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:14.224674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:14.224573    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:15.760956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:15.760922    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:16.969271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:16.969243    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:18.104028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:18.103998    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:21.792079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:21.792007    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:26.682956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:26.682922    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:30.053965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.053933    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"]
Nov 02 03:13:30.057592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.057574    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.110858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.110823    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-ca6546-openshift-default-dockercfg-c2jsr\""
Nov 02 03:13:30.132517 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132479    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.132735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132528    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.132735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132599    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aeab29-37eb-4500-bbb6-79543a341348-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.132735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132650    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.132735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132701    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.132932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132795    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.132932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132852    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.132932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132884    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aeab29-37eb-4500-bbb6-79543a341348-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.133069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.132955    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssk9l\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-kube-api-access-ssk9l\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.164306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.164276    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"]
Nov 02 03:13:30.234174 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234135    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234204    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234245    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234295    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234334    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aeab29-37eb-4500-bbb6-79543a341348-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234383    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssk9l\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-kube-api-access-ssk9l\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234447    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234685    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234704    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234689    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234746    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aeab29-37eb-4500-bbb6-79543a341348-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234787    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.234991 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.234971    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.235321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.235305    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aeab29-37eb-4500-bbb6-79543a341348-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.236517 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.236496    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.236890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.236873    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aeab29-37eb-4500-bbb6-79543a341348-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.294814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.294785    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssk9l\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-kube-api-access-ssk9l\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.294943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.294865    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.370650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.370552    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:30.792068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.792044    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"]
Nov 02 03:13:30.796180 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:13:30.795281    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0aeab29_37eb_4500_bbb6_79543a341348.slice/crio-382b6890e8b46e9ac162d493577d63d12b63a66986c1547b8caf0e0eeff2f754 WatchSource:0}: Error finding container 382b6890e8b46e9ac162d493577d63d12b63a66986c1547b8caf0e0eeff2f754: Status 404 returned error can't find the container with id 382b6890e8b46e9ac162d493577d63d12b63a66986c1547b8caf0e0eeff2f754
Nov 02 03:13:30.798394 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.798358    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:30.798495 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.798430    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:30.798495 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:30.798460    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:31.229747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:31.229660    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" event={"ID":"f0aeab29-37eb-4500-bbb6-79543a341348","Type":"ContainerStarted","Data":"ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937"}
Nov 02 03:13:31.229747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:31.229712    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" event={"ID":"f0aeab29-37eb-4500-bbb6-79543a341348","Type":"ContainerStarted","Data":"382b6890e8b46e9ac162d493577d63d12b63a66986c1547b8caf0e0eeff2f754"}
Nov 02 03:13:31.349487 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:31.349427    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podStartSLOduration=1.349413307 podStartE2EDuration="1.349413307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:13:31.347846706 +0000 UTC m=+1555.143195270" watchObservedRunningTime="2026-11-02 03:13:31.349413307 +0000 UTC m=+1555.144761871"
Nov 02 03:13:31.371721 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:31.371696    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:31.373163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:31.373140    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 03:13:31.373259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:31.373188    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 03:13:31.556417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:31.556388    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:32.371092 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:32.371062    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 03:13:32.371491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:32.371112    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 03:13:32.527706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:32.527672    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:13:33.371500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:33.371462    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 03:13:33.372055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:33.371522    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 03:13:34.371238 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:34.371201    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 03:13:34.371437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:34.371271    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 03:13:35.218858 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:13:35.218785    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:13:35.371030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:35.370996    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 03:13:35.371178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:35.371063    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 03:13:36.370920 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:36.370888    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 03:13:36.371294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:36.370951    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 03:13:36.504254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:36.504196    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:36.706568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:36.706464    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:36.873906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:36.873867    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:37.371963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:37.371926    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 03:13:37.372535 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:37.371992    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 03:13:38.475974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:38.475943    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:38.476462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:38.476383    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:38.477516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:38.477494    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 03:13:41.412688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:41.412651    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:46.340728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:46.340691    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:50.849216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.849127    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"]
Nov 02 03:13:50.854082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.854059    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878453 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878426    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878586 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878467    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878586 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878496    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878638    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878684    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878715    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hpb5\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-kube-api-access-8hpb5\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878794    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878816    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.878931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.878919    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.899831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.899807    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"]
Nov 02 03:13:50.903165 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.903148    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.951765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.951731    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"]
Nov 02 03:13:50.979968 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.979939    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.980146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.979978    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.980146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980014    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.980146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980047    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbk5j\" (UniqueName: \"kubernetes.io/projected/d890d798-0f37-4e21-a517-c382a56b9ca3-kube-api-access-hbk5j\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.980323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980220    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.980323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980266    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.980423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980341    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.980423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980403    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.980534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980445    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.980534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980475    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d890d798-0f37-4e21-a517-c382a56b9ca3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.980534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980505    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.980729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980656    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.980847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980830    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.980933 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980867    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.980933 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980906    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.981041 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980945    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.981041 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.980946    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.981151 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.981037    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.981151 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.981075    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.981151 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.981112    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hpb5\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-kube-api-access-8hpb5\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.981151 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.981139    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.981337 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.981162    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:50.981337 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.981199    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.983986 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.983944    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:50.984453 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:50.984428    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:51.002456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.002427    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"]
Nov 02 03:13:51.038483 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.038454    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:51.038822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.038799    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hpb5\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-kube-api-access-8hpb5\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5lq87\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:51.082093 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082055    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082105    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d890d798-0f37-4e21-a517-c382a56b9ca3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082132    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082152    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082173    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082236    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082251    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082308    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082326    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbk5j\" (UniqueName: \"kubernetes.io/projected/d890d798-0f37-4e21-a517-c382a56b9ca3-kube-api-access-hbk5j\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082516    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082562    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.082920 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082890    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.083028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.082969    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.083287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.083251    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d890d798-0f37-4e21-a517-c382a56b9ca3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.085448 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.085427    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.085875 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.085858    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.140060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.139981    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d890d798-0f37-4e21-a517-c382a56b9ca3-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.140196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.140085    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbk5j\" (UniqueName: \"kubernetes.io/projected/d890d798-0f37-4e21-a517-c382a56b9ca3-kube-api-access-hbk5j\") pod \"maas-default-gateway-openshift-default-58847f9fc5-nmpc6\" (UID: \"d890d798-0f37-4e21-a517-c382a56b9ca3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.166669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.166616    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:51.173994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.173970    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:51.215485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.215452    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:51.595160 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.595131    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"]
Nov 02 03:13:51.596645 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:13:51.596585    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a453c7c_8a6a_4171_8d2a_08217c1d8f7a.slice/crio-a44234b62d0955e74415c854aa03b85f6ec6e395009a891121bd0ae26dd4019b WatchSource:0}: Error finding container a44234b62d0955e74415c854aa03b85f6ec6e395009a891121bd0ae26dd4019b: Status 404 returned error can't find the container with id a44234b62d0955e74415c854aa03b85f6ec6e395009a891121bd0ae26dd4019b
Nov 02 03:13:51.598836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.598805    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:51.598904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.598871    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:51.598945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.598902    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:51.647122 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.647102    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"]
Nov 02 03:13:51.647933 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:13:51.647907    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd890d798_0f37_4e21_a517_c382a56b9ca3.slice/crio-7dd13678773a29ae65b6f81192b0923520209351f872611a0aaa195a530c204e WatchSource:0}: Error finding container 7dd13678773a29ae65b6f81192b0923520209351f872611a0aaa195a530c204e: Status 404 returned error can't find the container with id 7dd13678773a29ae65b6f81192b0923520209351f872611a0aaa195a530c204e
Nov 02 03:13:51.650726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.650687    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:51.650796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.650774    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:51.650853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:51.650839    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:13:52.357100 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:52.357066    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" event={"ID":"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a","Type":"ContainerStarted","Data":"d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b"}
Nov 02 03:13:52.357588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:52.357129    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" event={"ID":"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a","Type":"ContainerStarted","Data":"a44234b62d0955e74415c854aa03b85f6ec6e395009a891121bd0ae26dd4019b"}
Nov 02 03:13:52.358860 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:52.358834    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" event={"ID":"d890d798-0f37-4e21-a517-c382a56b9ca3","Type":"ContainerStarted","Data":"9611161d4709b0e32c12ea27db8d586d081c9deba8989d9f30544b54b5fcf17f"}
Nov 02 03:13:52.358971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:52.358884    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" event={"ID":"d890d798-0f37-4e21-a517-c382a56b9ca3","Type":"ContainerStarted","Data":"7dd13678773a29ae65b6f81192b0923520209351f872611a0aaa195a530c204e"}
Nov 02 03:13:52.477309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:52.476262    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podStartSLOduration=2.476239621 podStartE2EDuration="2.476239621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:13:52.47443042 +0000 UTC m=+1576.269778985" watchObservedRunningTime="2026-11-02 03:13:52.476239621 +0000 UTC m=+1576.271588186"
Nov 02 03:13:52.595651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:52.594429    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podStartSLOduration=2.5944130100000002 podStartE2EDuration="2.59441301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:13:52.594130261 +0000 UTC m=+1576.389478829" watchObservedRunningTime="2026-11-02 03:13:52.59441301 +0000 UTC m=+1576.389761575"
Nov 02 03:13:53.167761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:53.167725    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:13:53.168933 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:53.168914    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5lq87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 03:13:53.168993 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:53.168959    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 03:13:53.230579 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:53.230455    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:13:53.232087 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:53.232065    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-nmpc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 03:13:53.232276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:53.232242    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podUID="d890d798-0f37-4e21-a517-c382a56b9ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 03:13:54.167228 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:54.167196    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5lq87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 03:13:54.167615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:54.167262    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 03:13:54.217650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:54.216950    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-nmpc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 03:13:54.217650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:54.217011    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podUID="d890d798-0f37-4e21-a517-c382a56b9ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 03:13:54.726651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:54.725963    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:13:55.167118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:55.167082    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5lq87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 03:13:55.167322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:55.167141    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 03:13:55.219651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:55.219021    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-nmpc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 03:13:55.219651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:55.219082    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podUID="d890d798-0f37-4e21-a517-c382a56b9ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 03:13:56.086716 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:56.086686    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:13:56.167694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:56.167663    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5lq87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 03:13:56.168082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:56.167722    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 03:13:56.220068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:56.220031    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-nmpc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 03:13:56.220361 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:56.220333    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podUID="d890d798-0f37-4e21-a517-c382a56b9ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 03:13:57.167779 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:57.167743    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5lq87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 03:13:57.168211 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:57.167798    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 03:13:57.217997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:57.217964    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-nmpc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 03:13:57.218274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:57.218246    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podUID="d890d798-0f37-4e21-a517-c382a56b9ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 03:13:58.167797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:58.167757    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5lq87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 03:13:58.168267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:58.167821    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 03:13:58.216280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:58.216234    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-nmpc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 03:13:58.216645 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:58.216300    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podUID="d890d798-0f37-4e21-a517-c382a56b9ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 03:13:59.167228 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:59.167187    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5lq87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 03:13:59.167407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:59.167256    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 03:13:59.216309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:59.216276    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-nmpc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 02 03:13:59.216733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:13:59.216337    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podUID="d890d798-0f37-4e21-a517-c382a56b9ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 02 03:14:00.491503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:00.491466    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:14:00.492446 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:00.492397    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:14:00.492905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:00.492881    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6"
Nov 02 03:14:00.678523 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:00.678488    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:14:00.679200 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:00.679177    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:14:00.679865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:00.679841    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:14:00.876413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:00.876376    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:04.206332 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.206288    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"]
Nov 02 03:14:04.210323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.210301    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.263526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.263495    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-5ac93f-openshift-default-dockercfg-cjnld\""
Nov 02 03:14:04.316798 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.316769    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"]
Nov 02 03:14:04.400885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.400853    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t9p8\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-kube-api-access-9t9p8\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.401098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.400914    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b9b58-87b5-443e-908a-56c80a422a4e-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.401098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.401033    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.401098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.401072    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.401277 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.401199    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.401277 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.401261    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b9b58-87b5-443e-908a-56c80a422a4e-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.401380 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.401366    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.401442 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.401405    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.401497 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.401463    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.502639 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.502590    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b9b58-87b5-443e-908a-56c80a422a4e-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.502822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.502651    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.502822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.502683    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.502822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.502757    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.502822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.502810    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b9b58-87b5-443e-908a-56c80a422a4e-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.502866    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.502905    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.502934    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.503033    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t9p8\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-kube-api-access-9t9p8\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503257 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.503210    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503317 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.503269    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.503451    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.503467    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b9b58-87b5-443e-908a-56c80a422a4e-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.503581 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.503494    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.505134 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.505116    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.505401 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.505381    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b9b58-87b5-443e-908a-56c80a422a4e-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.560730 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.560690    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.560912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.560793    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t9p8\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-kube-api-access-9t9p8\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:04.823419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:04.823336    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:05.188750 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:14:05.188666    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:14:05.249896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.249872    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"]
Nov 02 03:14:05.251550 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:14:05.251525    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554b9b58_87b5_443e_908a_56c80a422a4e.slice/crio-2fb811c78422160937029711e59f7c506bb744f604bade242c2d6498ee266def WatchSource:0}: Error finding container 2fb811c78422160937029711e59f7c506bb744f604bade242c2d6498ee266def: Status 404 returned error can't find the container with id 2fb811c78422160937029711e59f7c506bb744f604bade242c2d6498ee266def
Nov 02 03:14:05.253827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.253793    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:14:05.253890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.253864    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:14:05.253927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.253892    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:14:05.457790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.457693    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" event={"ID":"554b9b58-87b5-443e-908a-56c80a422a4e","Type":"ContainerStarted","Data":"815ce265a1eb4aa57a1e02e635b0cbac12629a2bb2bba2af7b23eeb295a4de80"}
Nov 02 03:14:05.457790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.457760    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" event={"ID":"554b9b58-87b5-443e-908a-56c80a422a4e","Type":"ContainerStarted","Data":"2fb811c78422160937029711e59f7c506bb744f604bade242c2d6498ee266def"}
Nov 02 03:14:05.577053 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.577001    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podStartSLOduration=1.576985691 podStartE2EDuration="1.576985691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:14:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:14:05.574498841 +0000 UTC m=+1589.369847408" watchObservedRunningTime="2026-11-02 03:14:05.576985691 +0000 UTC m=+1589.372334255"
Nov 02 03:14:05.772473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.772444    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:05.823995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:05.823958    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:06.825368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:06.825337    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 03:14:06.825868 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:06.825384    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 03:14:07.824248 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:07.824210    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 03:14:07.824409 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:07.824269    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 03:14:08.824656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:08.824596    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 03:14:08.825135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:08.824699    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 03:14:09.824688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:09.824648    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 03:14:09.825109 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:09.824705    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 03:14:10.609075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:10.609030    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:10.824152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:10.824121    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 02 03:14:10.824327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:10.824172    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 02 03:14:11.922778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:11.922746    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:11.923231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:11.923161    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:11.923979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:11.923960    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:14:15.533186 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:15.533155    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:18.850755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:18.850713    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:20.274854 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:20.274820    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:24.184583 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:24.184537    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:25.119772 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:25.119736    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:26.441378 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:26.441342    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:14:29.944368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:29.944336    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:34.758519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:34.758483    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:35.188577 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:14:35.188494    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:14:35.623168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:35.623126    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:37.166273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:37.166246    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:40.050907 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:40.050863    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:44.871897 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:44.871859    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:46.784189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:46.784162    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:14:48.490390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:48.490345    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:48.645930 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:48.645899    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:48.829052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:48.829016    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:49.779090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:49.779062    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:53.458318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:53.458285    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:54.684473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:54.684442    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:14:59.468979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:14:59.468908    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:04.781524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:04.781473    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:05.188116 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:15:05.188018    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:15:09.577821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:09.577793    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:14.374783 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:14.374750    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:17.568253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:17.568221    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:15:19.203107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:19.203070    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:24.262356 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.262319    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"]
Nov 02 03:15:24.265957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.265943    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.270674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.270650    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:24.312179 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.312146    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tls-certs\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.312364 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.312265    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-model-cache\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.312364 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.312294    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-home\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.312364 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.312323    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfr9t\" (UniqueName: \"kubernetes.io/projected/7d7ee11e-a411-4249-ba0d-89e5bdc96760-kube-api-access-wfr9t\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.312364 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.312364    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tmp-dir\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.312592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.312412    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-dshm\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.327156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.327123    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"]
Nov 02 03:15:24.372600 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.372571    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1f6e39\"/\"distinct-2-w0-1f6e39-kserve-self-signed-certs\""
Nov 02 03:15:24.413522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.413474    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfr9t\" (UniqueName: \"kubernetes.io/projected/7d7ee11e-a411-4249-ba0d-89e5bdc96760-kube-api-access-wfr9t\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.414297 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.414205    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tmp-dir\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.414481 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.414307    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-dshm\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.414481 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.414416    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tls-certs\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.414615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.414543    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-model-cache\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.414615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.414585    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-home\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.415086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.415061    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-home\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.415666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.415594    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tmp-dir\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.416811 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.416789    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-model-cache\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.419552 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.419511    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-dshm\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.422324 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.422276    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tls-certs\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.471320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.471288    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfr9t\" (UniqueName: \"kubernetes.io/projected/7d7ee11e-a411-4249-ba0d-89e5bdc96760-kube-api-access-wfr9t\") pod \"distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") " pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.475006 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.474956    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:24.578458 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.578367    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:24.949662 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.949615    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"]
Nov 02 03:15:24.951574 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:15:24.951545    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d7ee11e_a411_4249_ba0d_89e5bdc96760.slice/crio-d27065d9eb4f71e3fe97abfa63abe1a4bf898d069651059b4a60710ad159a9aa WatchSource:0}: Error finding container d27065d9eb4f71e3fe97abfa63abe1a4bf898d069651059b4a60710ad159a9aa: Status 404 returned error can't find the container with id d27065d9eb4f71e3fe97abfa63abe1a4bf898d069651059b4a60710ad159a9aa
Nov 02 03:15:24.979124 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:24.979097    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" event={"ID":"7d7ee11e-a411-4249-ba0d-89e5bdc96760","Type":"ContainerStarted","Data":"d27065d9eb4f71e3fe97abfa63abe1a4bf898d069651059b4a60710ad159a9aa"}
Nov 02 03:15:25.986568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:25.986525    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" event={"ID":"7d7ee11e-a411-4249-ba0d-89e5bdc96760","Type":"ContainerStarted","Data":"7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef"}
Nov 02 03:15:26.102991 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:26.102933    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podStartSLOduration=2.102917753 podStartE2EDuration="2.102917753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:15:26.101171492 +0000 UTC m=+1669.896520058" watchObservedRunningTime="2026-11-02 03:15:26.102917753 +0000 UTC m=+1669.898266318"
Nov 02 03:15:26.579199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:26.579160    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:29.067176 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:29.067147    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:30.955192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:30.955163    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"]
Nov 02 03:15:30.958851 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:30.958832    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.064919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.064888    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"]
Nov 02 03:15:31.093505 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.093457    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.093792 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.093768    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdwvn\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-kube-api-access-xdwvn\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.093903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.093827    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.093994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.093969    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.094056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.094039    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b40fbd-084e-4593-aead-7c17531fb343-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.094192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.094177    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.094238 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.094224    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.094344 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.094325    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.094426 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.094401    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b40fbd-084e-4593-aead-7c17531fb343-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195419    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195668 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195462    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdwvn\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-kube-api-access-xdwvn\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195668 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195496    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195668 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195529    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195668 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195567    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b40fbd-084e-4593-aead-7c17531fb343-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195669    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195707    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195742    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.195910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195771    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b40fbd-084e-4593-aead-7c17531fb343-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.196104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.195912    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.196104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.196046    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.196228 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.196202    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.196470 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.196450    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b40fbd-084e-4593-aead-7c17531fb343-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.196566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.196447    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.198596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.198571    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b40fbd-084e-4593-aead-7c17531fb343-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.198714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.198606    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.253595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.253564    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.253861 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.253838    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdwvn\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-kube-api-access-xdwvn\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.273730 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.273705    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:31.702831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.702797    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"]
Nov 02 03:15:31.704057 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:15:31.704034    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b40fbd_084e_4593_aead_7c17531fb343.slice/crio-966479eddb5616a9cbdaa9cdeaa04e92a69f6757c9c1c68ec82abd67b8aa72af WatchSource:0}: Error finding container 966479eddb5616a9cbdaa9cdeaa04e92a69f6757c9c1c68ec82abd67b8aa72af: Status 404 returned error can't find the container with id 966479eddb5616a9cbdaa9cdeaa04e92a69f6757c9c1c68ec82abd67b8aa72af
Nov 02 03:15:31.706248 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.706213    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:15:31.706327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.706280    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:15:31.706327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:31.706306    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:15:32.021282 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:32.021232    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" event={"ID":"a7b40fbd-084e-4593-aead-7c17531fb343","Type":"ContainerStarted","Data":"2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68"}
Nov 02 03:15:32.021742 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:32.021303    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" event={"ID":"a7b40fbd-084e-4593-aead-7c17531fb343","Type":"ContainerStarted","Data":"966479eddb5616a9cbdaa9cdeaa04e92a69f6757c9c1c68ec82abd67b8aa72af"}
Nov 02 03:15:32.143361 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:32.143312    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podStartSLOduration=2.143296403 podStartE2EDuration="2.143296403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:15:32.140353097 +0000 UTC m=+1675.935701665" watchObservedRunningTime="2026-11-02 03:15:32.143296403 +0000 UTC m=+1675.938645448"
Nov 02 03:15:32.274300 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:32.274198    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:32.276368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:32.276341    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 02 03:15:32.276496 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:32.276395    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 02 03:15:33.274514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:33.274480    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 02 03:15:33.274969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:33.274535    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 02 03:15:34.011142 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:34.011105    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:34.274310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:34.274230    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 02 03:15:34.274310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:34.274285    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 02 03:15:34.578704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:34.578606    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:34.615971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:34.615940    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:35.047295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:35.047269    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:15:35.226643 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:15:35.225991    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:15:35.276093 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:35.275288    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 02 03:15:35.276093 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:35.275348    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 02 03:15:36.274982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:36.274937    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 02 03:15:36.275458 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:36.275069    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 02 03:15:37.274123 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:37.274086    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 02 03:15:37.274318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:37.274150    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 02 03:15:37.506964 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:37.506923    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:37.766332 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:37.766298    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:38.274699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:38.274667    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 02 03:15:38.274873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:38.274719    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 02 03:15:38.958416 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:38.958380    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:39.115991 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:39.115959    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:39.279072 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:39.279047    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:39.279322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:39.279302    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:39.280040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:39.280022    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:15:43.775672 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:43.775612    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:47.649267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:47.649231    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:48.688770 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:48.688733    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:49.437437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:49.437404    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:15:51.719357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.719324    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"]
Nov 02 03:15:51.724309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.724289    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.746314 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746290    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.746558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746343    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4zvh\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-kube-api-access-d4zvh\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.746558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746453    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.746558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746481    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6393d363-3485-4df1-ad45-781181c46561-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.746558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746511    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.747291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746682    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6393d363-3485-4df1-ad45-781181c46561-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.747291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746720    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.747291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746751    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.747291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.746806    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.830496 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.830465    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"]
Nov 02 03:15:51.848244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848215    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848281    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848316    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4zvh\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-kube-api-access-d4zvh\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848358    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848377    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6393d363-3485-4df1-ad45-781181c46561-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848598 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848397    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848598 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848447    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6393d363-3485-4df1-ad45-781181c46561-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848598 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848479    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848598 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848511    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.848832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848774    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.849068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.848972    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.849159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.849138    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6393d363-3485-4df1-ad45-781181c46561-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.849225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.849204    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.849264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.849235    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.850674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.850657    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.851066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.851049    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6393d363-3485-4df1-ad45-781181c46561-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.906178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.906146    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:51.906338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:51.906244    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4zvh\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-kube-api-access-d4zvh\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:52.036881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:52.036849    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:52.462516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:52.462488    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"]
Nov 02 03:15:52.463644 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:15:52.463592    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6393d363_3485_4df1_ad45_781181c46561.slice/crio-e2088963dbee2d791f0dfd434a309ccc9e2bcedd07321e445c5481cafc70f508 WatchSource:0}: Error finding container e2088963dbee2d791f0dfd434a309ccc9e2bcedd07321e445c5481cafc70f508: Status 404 returned error can't find the container with id e2088963dbee2d791f0dfd434a309ccc9e2bcedd07321e445c5481cafc70f508
Nov 02 03:15:52.465903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:52.465868    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:15:52.465968 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:52.465937    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:15:52.466016 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:52.465965    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:15:53.142712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:53.142670    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" event={"ID":"6393d363-3485-4df1-ad45-781181c46561","Type":"ContainerStarted","Data":"7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f"}
Nov 02 03:15:53.143113 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:53.142733    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" event={"ID":"6393d363-3485-4df1-ad45-781181c46561","Type":"ContainerStarted","Data":"e2088963dbee2d791f0dfd434a309ccc9e2bcedd07321e445c5481cafc70f508"}
Nov 02 03:15:53.362949 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:53.362899    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podStartSLOduration=2.3628847029999998 podStartE2EDuration="2.362884703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:15:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:15:53.311761698 +0000 UTC m=+1697.107110265" watchObservedRunningTime="2026-11-02 03:15:53.362884703 +0000 UTC m=+1697.158233267"
Nov 02 03:15:53.458020 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:53.457930    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:54.037075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:54.037043    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:15:54.038690 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:54.038666    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 02 03:15:54.038822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:54.038714    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 02 03:15:55.037245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:55.037204    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 02 03:15:55.037693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:55.037276    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 02 03:15:56.037187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:56.037151    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 02 03:15:56.037377 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:56.037201    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 02 03:15:57.037268 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:57.037236    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 02 03:15:57.037444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:57.037289    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 02 03:15:57.906663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:57.906500    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:58.037261 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:58.037233    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 02 03:15:58.037434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:58.037278    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 02 03:15:58.056000 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:58.055969    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:58.253102 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:58.253069    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:15:59.037294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:59.037250    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 02 03:15:59.037515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:15:59.037310    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 02 03:16:00.042071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:00.042031    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:16:00.042554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:00.042440    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:16:00.043280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:00.043262    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:16:01.377290 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:01.377256    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:01.535056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:01.535022    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:01.686885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:01.686794    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:03.228852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:03.228788    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:03.943154 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:03.943119    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:05.188845 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:16:05.188809    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:16:08.061853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:08.061825    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:12.865573 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:12.865544    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:13.333698 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:13.333665    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:16:15.503566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:15.503526    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:15.800753 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:15.800579    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:15.918650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:15.897713    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:16:15.992670 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:15.987667    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:16.019671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.012785    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 02 03:16:16.077743 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.076744    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Nov 02 03:16:16.121650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.118733    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:16:16.121650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.118810    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:16:16.547651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.543736    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-nmpc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:16:16.547651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.543799    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-nmpc6" podUID="d890d798-0f37-4e21-a517-c382a56b9ca3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:16:16.763719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.763683    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-gnf96 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:16:16.763928 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.763742    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:16:16.789648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.786697    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5lq87 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:16:16.789648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:16.786755    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:16:18.274030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:18.273997    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:23.107961 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:23.107912    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:28.221672 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:28.221633    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:33.178189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:33.178156    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:35.188600 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:16:35.188565    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:16:35.358897 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.358866    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"]
Nov 02 03:16:35.362600 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.362585    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.421404 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.421362    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"]
Nov 02 03:16:35.544383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544349    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.544561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544415    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.544561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544510    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.544561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544543    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.544729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544672    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.544729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544720    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.544804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544743    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.544804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544780    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45kqk\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-kube-api-access-45kqk\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.544906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.544887    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/587a7c84-880a-4726-bc56-bf1e47c2a98c-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.645873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.645837    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.645873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.645876    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.645901    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45kqk\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-kube-api-access-45kqk\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.645939    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/587a7c84-880a-4726-bc56-bf1e47c2a98c-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.645976    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646049    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646120    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646154    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646234    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646312    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646350    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646595    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646668    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.646927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.646904    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/587a7c84-880a-4726-bc56-bf1e47c2a98c-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.648664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.648616    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.648877 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.648858    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.704430 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.704396    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.704660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.704639    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45kqk\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-kube-api-access-45kqk\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:35.975867 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:35.975780    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:36.400260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:36.400230    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"]
Nov 02 03:16:36.401728 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:16:36.401707    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod587a7c84_880a_4726_bc56_bf1e47c2a98c.slice/crio-5994f0c05ad731a23e32c90cdeb8efb1bc7ca99b993aba89d06fbf03214b742a WatchSource:0}: Error finding container 5994f0c05ad731a23e32c90cdeb8efb1bc7ca99b993aba89d06fbf03214b742a: Status 404 returned error can't find the container with id 5994f0c05ad731a23e32c90cdeb8efb1bc7ca99b993aba89d06fbf03214b742a
Nov 02 03:16:36.404441 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:36.404410    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:36.404549 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:36.404476    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:36.404549 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:36.404506    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:36.520219 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:36.520183    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" event={"ID":"587a7c84-880a-4726-bc56-bf1e47c2a98c","Type":"ContainerStarted","Data":"5cce861eb307379ee7ad63bb176d097073a74d471a21e8592819315a0188921c"}
Nov 02 03:16:36.520403 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:36.520246    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" event={"ID":"587a7c84-880a-4726-bc56-bf1e47c2a98c","Type":"ContainerStarted","Data":"5994f0c05ad731a23e32c90cdeb8efb1bc7ca99b993aba89d06fbf03214b742a"}
Nov 02 03:16:36.639319 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:36.639264    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podStartSLOduration=1.6392491599999999 podStartE2EDuration="1.63924916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:16:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:16:36.637292046 +0000 UTC m=+1740.432640612" watchObservedRunningTime="2026-11-02 03:16:36.63924916 +0000 UTC m=+1740.434597724"
Nov 02 03:16:36.976764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:36.976736    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:37.070517 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.070484    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"]
Nov 02 03:16:37.074412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.074388    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.167159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.167129    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"]
Nov 02 03:16:37.210699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.209277    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:37.265294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265260    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.265514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265317    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqmpg\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-kube-api-access-hqmpg\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.265514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265350    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.265514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265366    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b1b168-b57f-437e-b114-512c12f4c753-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.265514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265409    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.265514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265437    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.265514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265470    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.265514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265492    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.265912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.265549    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b1b168-b57f-437e-b114-512c12f4c753-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366517 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366484    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366539    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqmpg\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-kube-api-access-hqmpg\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366579    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366602    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b1b168-b57f-437e-b114-512c12f4c753-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366692    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366733    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366781    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366816    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.366958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.366888    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b1b168-b57f-437e-b114-512c12f4c753-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.367169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.367092    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.367262 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.367242    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.367376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.367358    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.367475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.367457    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b1b168-b57f-437e-b114-512c12f4c753-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.367532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.367479    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.369023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.369001    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.369522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.369490    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b1b168-b57f-437e-b114-512c12f4c753-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.426580 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.426543    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.426972 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.426716    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqmpg\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-kube-api-access-hqmpg\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.705702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.704914    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:37.904660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.904482    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:37.977678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.977585    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 03:16:37.977678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:37.977653    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 03:16:38.134361 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.134327    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"]
Nov 02 03:16:38.134789 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:16:38.134766    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b1b168_b57f_437e_b114_512c12f4c753.slice/crio-aa78a65e5a8701905edb3ae21fec0a1740b7cf3473224efa7413669a433fdd94 WatchSource:0}: Error finding container aa78a65e5a8701905edb3ae21fec0a1740b7cf3473224efa7413669a433fdd94: Status 404 returned error can't find the container with id aa78a65e5a8701905edb3ae21fec0a1740b7cf3473224efa7413669a433fdd94
Nov 02 03:16:38.137441 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.137409    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:38.137541 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.137475    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:38.137541 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.137505    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:38.138603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.138575    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:38.533697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.533653    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" event={"ID":"a7b1b168-b57f-437e-b114-512c12f4c753","Type":"ContainerStarted","Data":"21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1"}
Nov 02 03:16:38.534129 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.533720    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" event={"ID":"a7b1b168-b57f-437e-b114-512c12f4c753","Type":"ContainerStarted","Data":"aa78a65e5a8701905edb3ae21fec0a1740b7cf3473224efa7413669a433fdd94"}
Nov 02 03:16:38.657503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.657448    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" podStartSLOduration=1.6574331500000001 podStartE2EDuration="1.65743315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:16:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:16:38.655062661 +0000 UTC m=+1742.450411242" watchObservedRunningTime="2026-11-02 03:16:38.65743315 +0000 UTC m=+1742.452781714"
Nov 02 03:16:38.706756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.706715    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:38.976690 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.976581    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 03:16:38.976690 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:38.976661    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 03:16:39.715917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:39.715878    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 03:16:39.716527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:39.716496    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 03:16:39.976603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:39.976515    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 03:16:39.976603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:39.976573    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 03:16:40.711652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:40.709041    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 03:16:40.711652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:40.709106    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 03:16:40.976062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:40.975982    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 03:16:40.976062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:40.976028    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 03:16:41.708088 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:41.708046    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 03:16:41.708382 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:41.708349    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 03:16:41.977098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:41.977018    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 03:16:41.977098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:41.977070    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 03:16:42.712649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:42.711982    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 03:16:42.712649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:42.712042    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 03:16:42.977099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:42.977012    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 02 03:16:42.977099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:42.977068    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 02 03:16:43.216533 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:43.216493    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:43.714656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:43.714129    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 03:16:43.714656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:43.714195    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 03:16:43.980345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:43.980267    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:43.980763 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:43.980535    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:44.006650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:43.994869    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:16:44.706978 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:44.706942    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 02 03:16:44.707185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:44.706992    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 02 03:16:45.711138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:45.711110    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:45.711574 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:45.711360    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:45.712241 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:45.712183    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:16:46.176614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:46.176587    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:16:48.420901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:48.420691    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:50.502898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.502865    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"]
Nov 02 03:16:50.507563 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.507539    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.645970    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsttz\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-kube-api-access-qsttz\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.646107    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.646141    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.646185    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.646273    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.646292    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.646324    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646546 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.646438    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.646546 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.646481    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.711290 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.711258    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"]
Nov 02 03:16:50.747580 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.747544    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.747800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.747598    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.747800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.747718    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsttz\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-kube-api-access-qsttz\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.747927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.747814    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.747927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.747848    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.747927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.747907    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.748082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.747946    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.748082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.747972    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.748082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.748016    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.748405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.748378    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.748804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.748777    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.748909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.748799    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.749066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.749043    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.749169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.749150    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.751186 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.751159    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.751555 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.751529    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.805229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.805160    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsttz\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-kube-api-access-qsttz\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.805457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.805435    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:50.821486 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:50.821462    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:51.250592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:51.250554    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"]
Nov 02 03:16:51.255815 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:16:51.255776    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a05cd9_4e74_402e_b6ff_dd3bb7a8caaf.slice/crio-79158fde7e56f55586a2838438c3798dadabd2b9cc2f5ab154be5fdb78e3c8d0 WatchSource:0}: Error finding container 79158fde7e56f55586a2838438c3798dadabd2b9cc2f5ab154be5fdb78e3c8d0: Status 404 returned error can't find the container with id 79158fde7e56f55586a2838438c3798dadabd2b9cc2f5ab154be5fdb78e3c8d0
Nov 02 03:16:51.258593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:51.258559    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:51.258680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:51.258660    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:51.258726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:51.258692    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:51.662863 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:51.662779    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" event={"ID":"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf","Type":"ContainerStarted","Data":"18cbc71793167f6dac55ad09632b1bd8881d370ea35b2babd9626a28390d223d"}
Nov 02 03:16:51.662863 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:51.662834    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" event={"ID":"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf","Type":"ContainerStarted","Data":"79158fde7e56f55586a2838438c3798dadabd2b9cc2f5ab154be5fdb78e3c8d0"}
Nov 02 03:16:51.785832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:51.785768    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podStartSLOduration=1.785747288 podStartE2EDuration="1.785747288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:16:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:16:51.783267745 +0000 UTC m=+1755.578616309" watchObservedRunningTime="2026-11-02 03:16:51.785747288 +0000 UTC m=+1755.581095854"
Nov 02 03:16:51.822309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:51.822277    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:52.206602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.206564    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"]
Nov 02 03:16:52.210918 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.210884    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.275315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.275277    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"]
Nov 02 03:16:52.368314 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368281    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.368314 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368317    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx7pg\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-kube-api-access-cx7pg\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.368532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368358    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.368532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368475    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.368532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368521    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.368727 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368709    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.368795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368775    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.368886 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368872    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.368955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.368940    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.469984 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.469880    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.469988    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470035    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470081    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470106    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx7pg\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-kube-api-access-cx7pg\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470180    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470249    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470351    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470479    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-certs\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470497    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470563    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470868    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-credential-socket\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.470969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.470893    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istiod-ca-cert\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.471104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.471081    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-data\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.473151 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.473127    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-envoy\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.473258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.473242    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-podinfo\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.528789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.528760    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-token\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.528948 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.528887    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx7pg\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-kube-api-access-cx7pg\") pod \"e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") " pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:52.823822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.823784    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 03:16:52.824232 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.823846    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 03:16:52.824995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:52.824976    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:53.274320 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:16:53.274286    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0573a2c_b964_43a7_aa5e_8d7f9baed082.slice/crio-8cca2a7071c6fd68fa642fecfbda69a119b1f7eccc7898e662c0a475bd774229 WatchSource:0}: Error finding container 8cca2a7071c6fd68fa642fecfbda69a119b1f7eccc7898e662c0a475bd774229: Status 404 returned error can't find the container with id 8cca2a7071c6fd68fa642fecfbda69a119b1f7eccc7898e662c0a475bd774229
Nov 02 03:16:53.274449 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.274367    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"]
Nov 02 03:16:53.277110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.277073    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:53.277230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.277149    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:53.277230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.277192    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:16:53.444036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.444003    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:53.676066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.675967    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" event={"ID":"e0573a2c-b964-43a7-aa5e-8d7f9baed082","Type":"ContainerStarted","Data":"74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8"}
Nov 02 03:16:53.676066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.676034    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" event={"ID":"e0573a2c-b964-43a7-aa5e-8d7f9baed082","Type":"ContainerStarted","Data":"8cca2a7071c6fd68fa642fecfbda69a119b1f7eccc7898e662c0a475bd774229"}
Nov 02 03:16:53.799931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.799865    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" podStartSLOduration=1.7998435860000002 podStartE2EDuration="1.799843586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:16:53.797133182 +0000 UTC m=+1757.592481748" watchObservedRunningTime="2026-11-02 03:16:53.799843586 +0000 UTC m=+1757.595192151"
Nov 02 03:16:53.822170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.822141    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 03:16:53.822338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.822203    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 03:16:53.825433 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:53.825406    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:16:54.841919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:54.837384    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 03:16:54.841919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:54.837455    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 03:16:54.843338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:54.843308    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 03:16:54.843640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:54.843579    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 03:16:54.859325 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:54.859279    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:55.830527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:55.830079    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 03:16:55.830527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:55.830137    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 03:16:55.831131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:55.831111    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 03:16:55.831279 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:55.831256    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 03:16:56.828106 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:56.828068    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 03:16:56.830662 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:56.830612    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 03:16:56.830822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:56.830549    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 03:16:56.830928 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:56.830911    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 03:16:57.822853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:57.822814    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 02 03:16:57.823060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:57.822878    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 02 03:16:57.826285 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:57.826261    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 03:16:57.826401 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:57.826308    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 03:16:58.578388 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:58.578356    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:58.737674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:58.737559    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:16:58.826090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:58.826048    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 03:16:58.826375 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:58.826345    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 03:16:58.924062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:58.924027    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:58.925413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:58.925393    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:58.932648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:58.931100    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:16:59.826035 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:59.826002    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 02 03:16:59.826595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:16:59.826059    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 02 03:17:00.830599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:00.830567    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:17:00.831076 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:00.830892    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:17:00.831559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:00.831542    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 03:17:03.568803 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:03.568772    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:05.189395 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:17:05.189363    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:17:08.657649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:08.654819    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:09.045014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:09.044969    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:09.502291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:09.501732    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:09.554511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:09.553668    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 02 03:17:09.743698 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:09.743650    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:09.905330 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:09.905229    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:10.045768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:10.045723    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:10.046784 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:10.046756    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:17:10.320714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:10.320682    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:17:13.875173 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:13.875134    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:15.544255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:15.544215    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:18.784335 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:18.784295    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:23.586802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:23.586754    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:26.299658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:26.298475    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:17:28.617652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:28.613768    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:28.746776 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:28.746724    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:28.891688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:28.891561    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:29.329779 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:29.329733    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:29.645709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:29.637103    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 02 03:17:29.645709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:29.638038    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:17:29.806221 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:29.806192    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:17:29.824896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:29.821931    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:29.824896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:29.822383    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:17:30.040023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.039981    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": context deadline exceeded"
Nov 02 03:17:30.041547 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.041496    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:17:30.043149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.043103    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:17:30.043331 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.043307    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:30.074134 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.072380    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:17:30.080778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.080733    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:17:30.081020 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.080999    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:30.088475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.088451    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:17:30.680246 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:30.680203    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:33.149857 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:33.149820    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:33.305128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:33.305067    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:35.190467 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:17:35.190416    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:17:35.705678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.705612    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:35.839582 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.839550    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"]
Nov 02 03:17:35.843810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.843781    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.903154 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.903120    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"]
Nov 02 03:17:35.925615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.925575    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.925615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.925615    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.925846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.925780    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9dp5\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-kube-api-access-p9dp5\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.925846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.925821    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.925846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.925840    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d358326-31d3-477c-85cc-5e5069e03e16-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.926031 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.926003    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.926086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.926053    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d358326-31d3-477c-85cc-5e5069e03e16-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.926165 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.926148    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:35.926267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:35.926251    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.027522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027486    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.027522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027522    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d358326-31d3-477c-85cc-5e5069e03e16-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.027793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027555    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.027793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027667    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.027793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027788    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.027957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027814    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.027957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027903    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9dp5\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-kube-api-access-p9dp5\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.027957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027952    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.028119 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.027976    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d358326-31d3-477c-85cc-5e5069e03e16-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.028119 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.028008    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.028119 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.028083    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.028264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.028132    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.028328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.028309    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.028379 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.028355    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d358326-31d3-477c-85cc-5e5069e03e16-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.030160 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.030137    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.030438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.030418    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d358326-31d3-477c-85cc-5e5069e03e16-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.085713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.085678    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9dp5\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-kube-api-access-p9dp5\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.085951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.085927    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.158169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.158133    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:36.587415 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.587388    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"]
Nov 02 03:17:36.588078 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:17:36.588052    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d358326_31d3_477c_85cc_5e5069e03e16.slice/crio-5323e7fc7cd8d28b76c167ad1907f0290f5ea64e72b3ae05a53980e4a8e74ffe WatchSource:0}: Error finding container 5323e7fc7cd8d28b76c167ad1907f0290f5ea64e72b3ae05a53980e4a8e74ffe: Status 404 returned error can't find the container with id 5323e7fc7cd8d28b76c167ad1907f0290f5ea64e72b3ae05a53980e4a8e74ffe
Nov 02 03:17:36.590515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.590481    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:36.590600 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.590559    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:36.590600 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:36.590587    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:37.200588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:37.200543    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" event={"ID":"3d358326-31d3-477c-85cc-5e5069e03e16","Type":"ContainerStarted","Data":"aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6"}
Nov 02 03:17:37.200821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:37.200680    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" event={"ID":"3d358326-31d3-477c-85cc-5e5069e03e16","Type":"ContainerStarted","Data":"5323e7fc7cd8d28b76c167ad1907f0290f5ea64e72b3ae05a53980e4a8e74ffe"}
Nov 02 03:17:37.321435 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:37.321374    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podStartSLOduration=2.32136054 podStartE2EDuration="2.32136054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:17:37.317951928 +0000 UTC m=+1801.113300497" watchObservedRunningTime="2026-11-02 03:17:37.32136054 +0000 UTC m=+1801.116709105"
Nov 02 03:17:38.159217 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:38.159181    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:38.160960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:38.160934    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 03:17:38.161081 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:38.160981    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 03:17:38.206587 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:38.206540    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:39.158880 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:39.158845    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 03:17:39.159086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:39.158894    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 03:17:40.159331 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:40.159298    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 03:17:40.159756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:40.159365    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 03:17:40.426826 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:40.426749    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:41.159656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:41.159025    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 03:17:41.159656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:41.159106    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 03:17:42.161649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:42.161142    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 03:17:42.161649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:42.161206    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 03:17:42.815648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:42.814781    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:42.975649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:42.974732    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:17:42.975649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:42.974807    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:42.995995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:42.995603    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 02 03:17:43.078666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:43.077740    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:43.093015 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:43.091731    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:43.161133 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:43.161089    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 03:17:43.161430 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:43.161396    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 03:17:43.348733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:43.347102    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:43.817768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:43.817715    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:43.819487 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:43.819462    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:17:43.959169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:43.959139    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:17:44.090423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.069791    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:44.090423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.070510    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:17:44.095058 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.092389    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:44.095058 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.092788    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:17:44.126565 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.126409    4407 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:44.201220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.199394    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 03:17:44.201220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.199454    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 03:17:44.243662 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.229691    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:17:44.365707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:44.340958    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:17:45.584545 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:45.584509    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:46.200773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:46.200741    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:17:46.201007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:46.200796    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:17:46.485046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:46.484949    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:46.485391 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:46.485365    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:46.486423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:46.486402    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:17:46.982400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:46.982367    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"]
Nov 02 03:17:46.986749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:46.986731    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033134 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033098    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033151    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033241    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033318    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9l7v\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-kube-api-access-b9l7v\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033478 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033340    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033478 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033423    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033478 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033443    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033478 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033479    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.033645 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.033596    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.045515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.045481    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"]
Nov 02 03:17:47.134461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134426    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.134694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134475    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.134694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134504    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.134694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134558    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9l7v\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-kube-api-access-b9l7v\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.134694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134586    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.134694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134657    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.134694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134680    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.135053 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134730    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.135053 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.134827    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.135168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.135101    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.135223 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.135169    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-data\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.135397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.135347    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istiod-ca-cert\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.135397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.135399    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-credential-socket\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.135657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.135402    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-certs\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.137229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.137207    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-envoy\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.137342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.137326    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-podinfo\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.192580 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.192547    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-token\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.193339 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.193314    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9l7v\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-kube-api-access-b9l7v\") pod \"e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") " pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.300879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.300843    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:47.730788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.730757    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"]
Nov 02 03:17:47.731837 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:17:47.731811    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e8e57a2_8970_4f43_ad20_ed3a3642268c.slice/crio-1fdb291d40f395b4a6a23196f2a540932c8d5bad271958f4dc660db3116b640a WatchSource:0}: Error finding container 1fdb291d40f395b4a6a23196f2a540932c8d5bad271958f4dc660db3116b640a: Status 404 returned error can't find the container with id 1fdb291d40f395b4a6a23196f2a540932c8d5bad271958f4dc660db3116b640a
Nov 02 03:17:47.734248 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.734216    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:47.734314 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.734282    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:47.734352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:47.734313    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:48.444943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:48.444905    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" event={"ID":"5e8e57a2-8970-4f43-ad20-ed3a3642268c","Type":"ContainerStarted","Data":"0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c"}
Nov 02 03:17:48.445403 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:48.444973    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" event={"ID":"5e8e57a2-8970-4f43-ad20-ed3a3642268c","Type":"ContainerStarted","Data":"1fdb291d40f395b4a6a23196f2a540932c8d5bad271958f4dc660db3116b640a"}
Nov 02 03:17:48.664995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:48.664945    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podStartSLOduration=2.664929712 podStartE2EDuration="2.664929712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:17:48.612189779 +0000 UTC m=+1812.407538343" watchObservedRunningTime="2026-11-02 03:17:48.664929712 +0000 UTC m=+1812.460278277"
Nov 02 03:17:49.305846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:49.305806    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:49.307753 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:49.307724    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 03:17:49.307910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:49.307786    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 03:17:50.302651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:50.302051    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 03:17:50.302651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:50.302117    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 03:17:50.474698 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:50.474664    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:51.302650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:51.302354    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 03:17:51.302650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:51.302430    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 03:17:52.306054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:52.306019    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 03:17:52.306496 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:52.306085    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 03:17:53.305789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:53.305754    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 03:17:53.305988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:53.305814    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 03:17:54.301817 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:54.301777    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 03:17:54.302206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:54.301842    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 03:17:55.302695 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:55.302665    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:55.305749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:55.305728    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:55.306130 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:55.306110    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:55.306865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:55.306847    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:17:56.984384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:56.984356    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:57.479685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.479645    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"]
Nov 02 03:17:57.483965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.483948    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.542020 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.541985    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d5dcca72-openshift-default-dockercfg-glcl8\""
Nov 02 03:17:57.616872 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.616832    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.617059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.616910    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.617059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.617033    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.617158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.617072    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.617158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.617117    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.617247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.617165    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.617292 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.617256    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.617338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.617308    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn9lg\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-kube-api-access-nn9lg\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.617380 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.617352    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.628304 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.628273    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:57.628614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.628592    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"]
Nov 02 03:17:57.730358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.730262    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.730358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.730327    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn9lg\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-kube-api-access-nn9lg\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.730596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.730367    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.730596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.730439    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.730596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.730478    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.730596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.730583    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.730833 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.730612    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.730989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.730956    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.731218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.731188    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.731312 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.731123    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.731312 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.731196    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.731417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.731380    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.731417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.731407    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.731666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.731613    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.733602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.733558    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.733991 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.733972    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.788460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.788422    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.788834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.788496    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn9lg\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-kube-api-access-nn9lg\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.798390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.798356    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:57.850876 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:57.850717    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:17:58.246225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:58.246200    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"]
Nov 02 03:17:58.247502 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:17:58.247467    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c79a00_a599_47d6_90f4_6e474cdf4d8a.slice/crio-9f2f878b27d56c136a234244cacb2664e7c8c7b4328eabaa454beadc921d9008 WatchSource:0}: Error finding container 9f2f878b27d56c136a234244cacb2664e7c8c7b4328eabaa454beadc921d9008: Status 404 returned error can't find the container with id 9f2f878b27d56c136a234244cacb2664e7c8c7b4328eabaa454beadc921d9008
Nov 02 03:17:58.249963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:58.249928    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:58.250051 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:58.249991    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:58.250051 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:58.250020    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:17:58.504436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:58.504405    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" event={"ID":"31c79a00-a599-47d6-90f4-6e474cdf4d8a","Type":"ContainerStarted","Data":"94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146"}
Nov 02 03:17:58.504594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:58.504460    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" event={"ID":"31c79a00-a599-47d6-90f4-6e474cdf4d8a","Type":"ContainerStarted","Data":"9f2f878b27d56c136a234244cacb2664e7c8c7b4328eabaa454beadc921d9008"}
Nov 02 03:17:58.625117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:58.625051    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" podStartSLOduration=1.625037021 podStartE2EDuration="1.625037021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:17:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:17:58.622433051 +0000 UTC m=+1822.417781616" watchObservedRunningTime="2026-11-02 03:17:58.625037021 +0000 UTC m=+1822.420385585"
Nov 02 03:17:58.799138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:58.799048    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:17:59.805305 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:17:59.805277    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:18:00.165746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:00.165657    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:00.516243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:00.516217    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:18:00.517178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:00.517160    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:18:01.446123 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:01.446088    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:01.633507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:01.633477    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:01.806534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:01.806494    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:02.227961 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:02.227853    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:02.389557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:02.389527    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:02.742861 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:02.742827    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:18:03.475124 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:03.475090    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:03.736090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:03.735991    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:04.739767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:04.739717    4407 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:05.035653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:05.022268    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 02 03:18:05.087178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:05.086728    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:05.129648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:05.114051    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:05.333745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:05.333165    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:05.337808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:05.337769    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:05.366692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:05.366644    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:05.460283 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:05.460179    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:05.601456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:05.596788    4407 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:05.899776 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:18:05.898312    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:18:06.042198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.042152    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:06.042745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.042724    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:18:06.087806 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.087738    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:06.088980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.088956    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:18:06.385476 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.385415    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:06.453358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.452552    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Nov 02 03:18:06.464202 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.464162    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 02 03:18:06.966358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.966320    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:18:06.966864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.966403    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:18:06.966864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.966437    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:18:06.966864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:06.966463    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 03:18:07.056792 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:07.056725    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:07.134674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:07.132830    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": context deadline exceeded"
Nov 02 03:18:07.459647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:07.445811    4407 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:07.459647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:07.446125    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:07.460264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:07.460224    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:08.059650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:08.058699    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 02 03:18:08.168995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:08.157700    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:08.183969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:08.171968    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:08.282177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:08.282130    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:08.586649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:08.581844    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": context deadline exceeded"
Nov 02 03:18:08.613658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:08.604924    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:09.074967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:09.074924    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:09.112654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:09.110028    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 03:18:09.116649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:09.116023    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 03:18:09.152274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:09.152231    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 03:18:09.280460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:09.280432    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 03:18:13.314500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:13.314467    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:17.378673 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:17.378638    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:18.199349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:18.199258    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:21.314975 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:21.314946    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:23.081273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:23.081246    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:26.141965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.141936    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"]
Nov 02 03:18:26.146145 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.146126    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.259362 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.259321    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"]
Nov 02 03:18:26.289506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.289457    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-certs\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.289731 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.289691    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gxlj\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-kube-api-access-4gxlj\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.289789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.289740    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-istio-token\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.289847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.289822    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-envoy\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.289904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.289891    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1606ccff-f513-4084-920f-38d8b883f29e-istio-podinfo\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.290039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.290016    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.290104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.290083    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-data\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.290247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.290230    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-credential-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.290283 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.290267    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1606ccff-f513-4084-920f-38d8b883f29e-istiod-ca-cert\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.391034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.391002    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gxlj\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-kube-api-access-4gxlj\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.391034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.391047    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-istio-token\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.391259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.391081    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-envoy\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.391259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.391157    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1606ccff-f513-4084-920f-38d8b883f29e-istio-podinfo\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.391259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.391183    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.391259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.391207    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-data\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.391391 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.391308    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-credential-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.391391 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.391336    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1606ccff-f513-4084-920f-38d8b883f29e-istiod-ca-cert\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.392215 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.392148    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1606ccff-f513-4084-920f-38d8b883f29e-istiod-ca-cert\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.392215 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.392208    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-certs\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.392715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.392691    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.393614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.393234    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-certs\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.393829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.393537    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-credential-socket\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.393989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.392157    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-data\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.396677 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.396247    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-envoy\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.396677 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.396603    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1606ccff-f513-4084-920f-38d8b883f29e-istio-podinfo\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.449492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.449450    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-istio-token\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.449664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.449567    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gxlj\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-kube-api-access-4gxlj\") pod \"e2e-worker-w4-398828-openshift-default-5647988965-bjxc6\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") " pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.461508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.461479    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:26.909029 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.909005    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"]
Nov 02 03:18:26.909577 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:18:26.909545    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1606ccff_f513_4084_920f_38d8b883f29e.slice/crio-f7eade6daeace28efc37011e46ccc2dc5eb348db6eb892e82730c1ec94ef3a20 WatchSource:0}: Error finding container f7eade6daeace28efc37011e46ccc2dc5eb348db6eb892e82730c1ec94ef3a20: Status 404 returned error can't find the container with id f7eade6daeace28efc37011e46ccc2dc5eb348db6eb892e82730c1ec94ef3a20
Nov 02 03:18:26.912318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.912282    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:18:26.912510 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.912488    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:18:26.912590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.912549    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:18:26.996567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:26.995157    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" event={"ID":"1606ccff-f513-4084-920f-38d8b883f29e","Type":"ContainerStarted","Data":"f7eade6daeace28efc37011e46ccc2dc5eb348db6eb892e82730c1ec94ef3a20"}
Nov 02 03:18:27.800529 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:27.800496    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:28.033231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:28.032291    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" event={"ID":"1606ccff-f513-4084-920f-38d8b883f29e","Type":"ContainerStarted","Data":"9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12"}
Nov 02 03:18:28.291650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:28.287950    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podStartSLOduration=2.287926229 podStartE2EDuration="2.287926229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:18:28.209148836 +0000 UTC m=+1852.004497406" watchObservedRunningTime="2026-11-02 03:18:28.287926229 +0000 UTC m=+1852.083274796"
Nov 02 03:18:28.535649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:28.535563    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:28.723961 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:28.714742    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 03:18:28.723961 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:28.714801    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 03:18:28.738693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:28.724926    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:29.241649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:29.239755    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:18:29.464301 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:29.464262    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 03:18:29.464596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:29.464568    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 03:18:30.462005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:30.461972    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 03:18:30.462649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:30.462577    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 03:18:31.457897 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:31.457865    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:31.462318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:31.462290    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 03:18:31.462700 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:31.462346    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 03:18:31.624324 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:31.624296    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:32.461932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:32.461896    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 03:18:32.462104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:32.461964    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 03:18:33.462554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:33.462520    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 03:18:33.463067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:33.462579    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 03:18:33.922686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:33.922648    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"]
Nov 02 03:18:33.928839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:33.928814    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:33.981741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:33.981709    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d5dcca72-serving-cert\""
Nov 02 03:18:34.012305 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.012266    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.012522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.012427    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg9xc\" (UniqueName: \"kubernetes.io/projected/35c59f44-c65d-41d7-932b-70d82d0dc930-kube-api-access-fg9xc\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.012522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.012502    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c59f44-c65d-41d7-932b-70d82d0dc930-openshift-service-ca\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.012732 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.012711    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-metrics-certs\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.037902 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.037864    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"]
Nov 02 03:18:34.114152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.114119    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg9xc\" (UniqueName: \"kubernetes.io/projected/35c59f44-c65d-41d7-932b-70d82d0dc930-kube-api-access-fg9xc\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.114336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.114194    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c59f44-c65d-41d7-932b-70d82d0dc930-openshift-service-ca\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.114336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.114272    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-metrics-certs\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.114456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.114335    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.114518 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:18:34.114489    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-serving-cert: secret "maas-api-e2e-mt-d5dcca72-serving-cert" not found
Nov 02 03:18:34.114575 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:18:34.114549    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls podName:35c59f44-c65d-41d7-932b-70d82d0dc930 nodeName:}" failed. No retries permitted until 2026-11-02 03:18:34.61452886 +0000 UTC m=+1858.409877418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls") pod "maas-api-e2e-mt-d5dcca72-565b874c9b-r628v" (UID: "35c59f44-c65d-41d7-932b-70d82d0dc930") : secret "maas-api-e2e-mt-d5dcca72-serving-cert" not found
Nov 02 03:18:34.115047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.115011    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c59f44-c65d-41d7-932b-70d82d0dc930-openshift-service-ca\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.116927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.116910    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-metrics-certs\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.157755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.157721    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:34.175323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.175250    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg9xc\" (UniqueName: \"kubernetes.io/projected/35c59f44-c65d-41d7-932b-70d82d0dc930-kube-api-access-fg9xc\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.323818 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.323782    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:34.486147 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.486048    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:34.621503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.621458    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.624254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.624227    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls\") pod \"maas-api-e2e-mt-d5dcca72-565b874c9b-r628v\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.843515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.843479    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:34.898593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.898560    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:34.899469 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.899414    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:34.900585 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:34.900553    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:18:35.198747 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:18:35.198639    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:18:35.259413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:35.259383    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"]
Nov 02 03:18:35.261216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:35.261191    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 03:18:36.180395 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:36.180353    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v" event={"ID":"35c59f44-c65d-41d7-932b-70d82d0dc930","Type":"ContainerStarted","Data":"e2bbbb67ec6262b6e73272ab4cf875084618e6e12b4be843456ad2c82fb1bc9a"}
Nov 02 03:18:37.188805 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:37.188750    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v" event={"ID":"35c59f44-c65d-41d7-932b-70d82d0dc930","Type":"ContainerStarted","Data":"1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d"}
Nov 02 03:18:37.189250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:37.188841    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:37.305360 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:37.305308    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v" podStartSLOduration=3.414167028 podStartE2EDuration="4.305293775s" totalImagesPullingTime="891.126747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:18:37.303140221 +0000 UTC m=+1861.098488787" watchObservedRunningTime="2026-11-02 03:18:37.305293775 +0000 UTC m=+1861.100642340"
Nov 02 03:18:38.541802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:38.541740    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:39.269697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:39.269652    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:42.311935 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:42.311891    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:42.614682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:42.614557    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:43.199674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:43.199642    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:18:44.196607 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:44.196575    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:49.102208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:49.102077    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:53.862177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:53.862134    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:58.741200 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:58.741097    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:59.049011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.048963    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:18:59.802496 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.802452    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"]
Nov 02 03:18:59.811994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.811965    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:18:59.857258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.857230    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"]
Nov 02 03:18:59.907485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.907454    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ca6546\"/\"default-dockercfg-cm7t6\""
Nov 02 03:18:59.908285 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.908258    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:18:59.908548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.908438    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ca6546\"/\"kube-root-ca.crt\""
Nov 02 03:18:59.916220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.916195    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ca6546\"/\"distinct-2-w2-ca6546-kserve-self-signed-certs\""
Nov 02 03:18:59.916382 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.916264    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ca6546\"/\"openshift-service-ca.crt\""
Nov 02 03:18:59.963368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.963327    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6zgb\" (UniqueName: \"kubernetes.io/projected/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-kube-api-access-p6zgb\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:18:59.963519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.963452    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-home\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:18:59.963519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.963505    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tmp-dir\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:18:59.963649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.963542    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tls-certs\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:18:59.963703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.963684    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-model-cache\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:18:59.963821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:18:59.963800    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-dshm\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.066853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.066756    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-dshm\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.067185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.067163    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6zgb\" (UniqueName: \"kubernetes.io/projected/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-kube-api-access-p6zgb\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.067390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.067372    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-home\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.067540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.067525    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tmp-dir\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.067700 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.067685    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tls-certs\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.099795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.099759    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-model-cache\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.100643 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.069010    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tmp-dir\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.100838 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.100427    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-model-cache\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.100948 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.068792    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-home\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.123649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.112969    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tls-certs\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.123649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.117978    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-dshm\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.148648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.131989    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6zgb\" (UniqueName: \"kubernetes.io/projected/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-kube-api-access-p6zgb\") pod \"distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") " pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.427980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.427881    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:00.961190 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:00.961002    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"]
Nov 02 03:19:01.377648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:01.370136    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj" event={"ID":"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2","Type":"ContainerStarted","Data":"1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac"}
Nov 02 03:19:01.377648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:01.370197    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj" event={"ID":"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2","Type":"ContainerStarted","Data":"b6f98a2db77c163712ad84820d47fabbb8cc1887d2f7b967f096772e6928e0fd"}
Nov 02 03:19:01.430647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:01.430510    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:01.504649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:01.496227    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj" podStartSLOduration=2.496209027 podStartE2EDuration="2.496209027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:18:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:19:01.495020249 +0000 UTC m=+1885.290368815" watchObservedRunningTime="2026-11-02 03:19:01.496209027 +0000 UTC m=+1885.291557591"
Nov 02 03:19:03.903328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:03.903222    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:04.105678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:04.099697    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:05.214805 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:19:05.214766    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:19:05.599114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.599075    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"]
Nov 02 03:19:05.603322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.603303    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.702580 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.702537    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"]
Nov 02 03:19:05.774438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.774352    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.774438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.774400    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.774611 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.774497    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv729\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-kube-api-access-lv729\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.774611 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.774534    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.774705 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.774665    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.774939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.774918    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.774993 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.774976    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.775053 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.775038    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.775098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.775072    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876043 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876009    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876050    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876103    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876131    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876227    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876251    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876280    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv729\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-kube-api-access-lv729\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876314    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876574 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876363    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876614    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876711    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.876997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.876975    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.877106 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.877078    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.877249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.877227    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.878983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.878957    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.879103 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.879084    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.934855 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.934821    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:05.935041 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:05.934896    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv729\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-kube-api-access-lv729\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:06.217966 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:06.217888    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:06.652873 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:19:06.652837    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17d8e32_b682_4c2e_8d56_b5aecc10042b.slice/crio-378fb0e8b3946e3a267f51cdbef476dd0bdfce5644d09dafcef786e3311f4abb WatchSource:0}: Error finding container 378fb0e8b3946e3a267f51cdbef476dd0bdfce5644d09dafcef786e3311f4abb: Status 404 returned error can't find the container with id 378fb0e8b3946e3a267f51cdbef476dd0bdfce5644d09dafcef786e3311f4abb
Nov 02 03:19:06.653227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:06.653198    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"]
Nov 02 03:19:06.655693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:06.655652    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:06.655782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:06.655743    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:06.655820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:06.655786    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:07.473266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:07.473227    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" event={"ID":"c17d8e32-b682-4c2e-8d56-b5aecc10042b","Type":"ContainerStarted","Data":"a18c4dd741414da31a1193ef14892af8baf1515aca26a385fbd0537ac5bf0f65"}
Nov 02 03:19:07.473758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:07.473301    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" event={"ID":"c17d8e32-b682-4c2e-8d56-b5aecc10042b","Type":"ContainerStarted","Data":"378fb0e8b3946e3a267f51cdbef476dd0bdfce5644d09dafcef786e3311f4abb"}
Nov 02 03:19:07.595659 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:07.592107    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" podStartSLOduration=2.59208772 podStartE2EDuration="2.59208772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:19:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:19:07.59084371 +0000 UTC m=+1891.386192294" watchObservedRunningTime="2026-11-02 03:19:07.59208772 +0000 UTC m=+1891.387436286"
Nov 02 03:19:08.218831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:08.218795    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:08.220669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:08.220640    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 03:19:08.220797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:08.220687    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 03:19:08.925138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:08.924521    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:09.218474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:09.218385    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 03:19:09.218474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:09.218439    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 03:19:09.799372 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:09.799263    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:10.218761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.218731    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 03:19:10.219191 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.218800    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 03:19:10.429221 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.429187    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:10.441991 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.441962    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:10.522530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.522495    4407 generic.go:356] "Generic (PLEG): container finished" podID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerID="94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146" exitCode=0
Nov 02 03:19:10.522715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.522568    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" event={"ID":"31c79a00-a599-47d6-90f4-6e474cdf4d8a","Type":"ContainerDied","Data":"94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146"}
Nov 02 03:19:10.523083 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.523065    4407 scope.go:122] "RemoveContainer" containerID="94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146"
Nov 02 03:19:10.524746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.524715    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:10.524812 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.524798    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:10.524858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.524843    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:10.534999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:10.534982    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 03:19:11.219279 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:11.219243    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 03:19:11.219737 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:11.219324    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 03:19:11.543135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:11.543096    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" event={"ID":"31c79a00-a599-47d6-90f4-6e474cdf4d8a","Type":"ContainerStarted","Data":"070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019"}
Nov 02 03:19:11.815164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:11.815132    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:19:11.832530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:11.832495    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 03:19:11.832793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:11.832764    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 03:19:12.228366 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:12.228273    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 03:19:12.228996 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:12.228965    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 03:19:12.800064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:12.800021    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 03:19:12.800415 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:12.800385    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 03:19:13.218417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:13.218323    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 02 03:19:13.218417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:13.218400    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 02 03:19:13.799107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:13.799067    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 03:19:13.799747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:13.799713    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 03:19:13.895146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:13.895111    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:14.362709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:14.362673    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:14.363240 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:14.363188    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:14.364375 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:14.364356    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:19:14.826171 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:14.826131    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 03:19:14.838673 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:14.837717    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 03:19:15.521649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:15.520442    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:19:15.800531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:15.800010    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 03:19:15.800531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:15.800067    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 03:19:16.800650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:16.798954    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 03:19:16.800650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:16.799011    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 03:19:17.624350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:17.624267    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:17.803273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:17.803231    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:19:17.804420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:17.804398    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:19:19.064254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:19.064212    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:23.941938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:23.941904    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:26.016944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:26.016909    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:26.181400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:26.181369    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:28.798836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:28.798805    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:29.417238 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:29.417204    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:19:33.187561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:33.187528    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:33.488695 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:33.488595    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:35.189922 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:19:35.189889    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:19:38.291966 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:38.291935    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:38.883070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:38.883037    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:43.116686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:43.116653    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:47.872922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:47.872892    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:51.264408 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:51.264293    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:51.790660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:51.789732    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"]
Nov 02 03:19:51.823664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:51.817994    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:51.888216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:51.888175    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"]
Nov 02 03:19:51.917746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:51.917587    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:51.946737 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:51.946694    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ed15c1-openshift-default-dockercfg-5dzdn\""
Nov 02 03:19:52.051711 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.022568    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-socket\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.052077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.052046    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9xck\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-kube-api-access-q9xck\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.052234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.052213    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-credential-socket\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.124651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.052381    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-podinfo\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.124651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.052444    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-envoy\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.124651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.052590    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istiod-ca-cert\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.124651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.052663    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-data\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.124651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.052757    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-token\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.124651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.052815    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-certs\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158002    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-socket\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158095    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9xck\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-kube-api-access-q9xck\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158122    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-credential-socket\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158170    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-podinfo\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158219    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-envoy\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158328    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istiod-ca-cert\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158369    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-data\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158445    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-token\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158492    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-certs\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.158958    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-certs\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.159294    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-socket\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.168687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.159861    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-credential-socket\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.200650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.196512    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-podinfo\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.267796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.217104    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-envoy\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.267796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.217858    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istiod-ca-cert\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.267796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.218108    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-data\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.369475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.369388    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9xck\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-kube-api-access-q9xck\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.468086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.468020    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-token\") pod \"e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") " pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:52.671355 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.671254    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:52.758650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:52.756905    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:53.134464 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.134427    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:53.326863 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.326833    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"]
Nov 02 03:19:53.327921 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:19:53.327895    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42b386b_9a17_4c21_b5f4_7cc0934d358a.slice/crio-feb3c970cea72a181be83aedfa8865847c42cf7a6c7dc7ee1add0c6848d8cb97 WatchSource:0}: Error finding container feb3c970cea72a181be83aedfa8865847c42cf7a6c7dc7ee1add0c6848d8cb97: Status 404 returned error can't find the container with id feb3c970cea72a181be83aedfa8865847c42cf7a6c7dc7ee1add0c6848d8cb97
Nov 02 03:19:53.330557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.330433    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:53.330648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.330615    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:53.330698 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.330668    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:53.624857 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.624778    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"]
Nov 02 03:19:53.628871 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.628853    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.681751    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-data\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.681832    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-socket\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.681878    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-token\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.681954    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-envoy\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.681991    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lzd9\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-kube-api-access-6lzd9\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.682034    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-certs\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.682121    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istiod-ca-cert\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.682147    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-credential-socket\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.703650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.682190    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-podinfo\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.762782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.744586    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"]
Nov 02 03:19:53.762782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.746074    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:53.797188 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.797146    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0d26af-openshift-default-dockercfg-gk25w\""
Nov 02 03:19:53.821717 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.821669    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-data\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.821900 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.821753    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-socket\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.821900 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.821796    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-token\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.821900 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.821871    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-envoy\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.822070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.821906    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lzd9\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-kube-api-access-6lzd9\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.822070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.821947    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-certs\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.822070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.822032    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istiod-ca-cert\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.822070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.822060    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-credential-socket\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.822281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.822104    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-podinfo\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.825115 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.825083    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-credential-socket\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.827558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.825953    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-socket\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.827558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.826711    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istiod-ca-cert\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.827558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.826949    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-certs\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.827844 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.827824    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-data\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.829714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.828412    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-podinfo\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.839493 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.839424    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-envoy\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.892714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.891231    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-token\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.895730 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.895696    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lzd9\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-kube-api-access-6lzd9\") pod \"e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") " pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.946321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.946291    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:53.951596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.951563    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" event={"ID":"c42b386b-9a17-4c21-b5f4-7cc0934d358a","Type":"ContainerStarted","Data":"3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c"}
Nov 02 03:19:53.951762 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:53.951645    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" event={"ID":"c42b386b-9a17-4c21-b5f4-7cc0934d358a","Type":"ContainerStarted","Data":"feb3c970cea72a181be83aedfa8865847c42cf7a6c7dc7ee1add0c6848d8cb97"}
Nov 02 03:19:54.043842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.043802    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:54.075165 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.075107    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" podStartSLOduration=3.075087152 podStartE2EDuration="3.075087152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:19:54.070784095 +0000 UTC m=+1937.866132673" watchObservedRunningTime="2026-11-02 03:19:54.075087152 +0000 UTC m=+1937.870435719"
Nov 02 03:19:54.196902 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.196823    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:54.383465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.383432    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"]
Nov 02 03:19:54.384399 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:19:54.384374    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95248c7_bf6d_4b12_9fa4_9c0b1fc1537b.slice/crio-56bf2eb09d08b0981251181434cee456ac011e7d7123d2736ecd8f8b9b0b902b WatchSource:0}: Error finding container 56bf2eb09d08b0981251181434cee456ac011e7d7123d2736ecd8f8b9b0b902b: Status 404 returned error can't find the container with id 56bf2eb09d08b0981251181434cee456ac011e7d7123d2736ecd8f8b9b0b902b
Nov 02 03:19:54.388243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.388191    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:54.388358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.388285    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:54.388358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.388325    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:19:54.757801 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.757768    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:54.763117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.763091    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:54.961301 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.961242    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" event={"ID":"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b","Type":"ContainerStarted","Data":"88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01"}
Nov 02 03:19:54.961301 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.961300    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" event={"ID":"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b","Type":"ContainerStarted","Data":"56bf2eb09d08b0981251181434cee456ac011e7d7123d2736ecd8f8b9b0b902b"}
Nov 02 03:19:54.961580 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.961540    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:54.962615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:54.962598    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:19:55.084273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:55.084160    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" podStartSLOduration=2.084141087 podStartE2EDuration="2.084141087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:19:55.083029942 +0000 UTC m=+1938.878378510" watchObservedRunningTime="2026-11-02 03:19:55.084141087 +0000 UTC m=+1938.879489653"
Nov 02 03:19:55.947067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:55.947030    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:55.953143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:55.953113    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:55.968674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:55.968646    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:55.969606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:55.969587    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:19:57.088850 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:57.088822    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:57.434018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:57.433931    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:58.067468 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:58.067437    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:58.297323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:58.297294    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:58.453201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:58.453129    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:19:59.338666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:19:59.338609    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:00.334847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:00.334812    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:00.518784 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:00.518751    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:02.923178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:02.923143    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:04.510689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:04.510660    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:05.185162 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:05.185128    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:05.188861 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:20:05.188834    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:20:05.355110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:05.355076    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:09.074105 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:09.074073    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:09.506526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:09.506495    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:12.472998 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:12.472964    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:20:14.736650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:14.727658    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:19.903091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:19.903057    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:21.192699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.192659    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"]
Nov 02 03:20:21.199481 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.199446    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272085 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272050    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcs7l\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-kube-api-access-mcs7l\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272094    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272128    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272259    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272359    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272400    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272440    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272672    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.272798 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.272778    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.307128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.307093    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"]
Nov 02 03:20:21.374384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374351    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcs7l\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-kube-api-access-mcs7l\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374383    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374404    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374526    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374587    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374638    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374669    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374795    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374869    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.374959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374890    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.375085 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.374961    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.375291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.375272    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.375539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.375289    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.375604 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.375530    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.376849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.376829    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.377275 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.377255    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.432094 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.432062    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.432271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.432248    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcs7l\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-kube-api-access-mcs7l\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.513340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.513306    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:21.942280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.942251    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"]
Nov 02 03:20:21.943206 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:20:21.943168    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4cd0ea3_6d06_47f1_98a2_a4f94645a44a.slice/crio-761c756e3231f8e0b0214033d878d59d59655de3f4c253336fc46dff8e4f2564 WatchSource:0}: Error finding container 761c756e3231f8e0b0214033d878d59d59655de3f4c253336fc46dff8e4f2564: Status 404 returned error can't find the container with id 761c756e3231f8e0b0214033d878d59d59655de3f4c253336fc46dff8e4f2564
Nov 02 03:20:21.945847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.945812    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:20:21.945938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.945879    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:20:21.945938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:21.945908    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:20:22.297765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:22.297726    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" event={"ID":"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a","Type":"ContainerStarted","Data":"56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784"}
Nov 02 03:20:22.298184 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:22.297798    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" event={"ID":"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a","Type":"ContainerStarted","Data":"761c756e3231f8e0b0214033d878d59d59655de3f4c253336fc46dff8e4f2564"}
Nov 02 03:20:22.418457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:22.418397    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" podStartSLOduration=1.4183820329999999 podStartE2EDuration="1.418382033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:20:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:20:22.416208043 +0000 UTC m=+1966.211556608" watchObservedRunningTime="2026-11-02 03:20:22.418382033 +0000 UTC m=+1966.213730598"
Nov 02 03:20:22.513542 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:22.513502    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:23.514921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:23.514884    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 03:20:23.515310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:23.514939    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 03:20:24.513996 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:24.513952    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 03:20:24.514196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:24.514024    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 03:20:24.713448 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:24.713410    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:25.513994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:25.513965    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 03:20:25.514184 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:25.514015    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 03:20:26.514164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:26.514121    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 03:20:26.514554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:26.514204    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 03:20:27.209997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:27.209959    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:27.405652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:27.405127    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:27.514480 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:27.514447    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 03:20:27.514908 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:27.514499    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 03:20:28.514544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:28.514507    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 02 03:20:28.514942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:28.514574    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 02 03:20:29.518013 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:29.517979    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:29.518447 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:29.518336    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:29.519008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:29.518992    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:20:29.664703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:29.664669    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:34.096187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.096152    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"]
Nov 02 03:20:34.100201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.100185    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.156904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.156869    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"]
Nov 02 03:20:34.167591 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.167559    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2zl2\" (UniqueName: \"kubernetes.io/projected/60e5f97c-02aa-4d35-abed-284be1654094-kube-api-access-s2zl2\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.167757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.167618    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.167966 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.167940    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e5f97c-02aa-4d35-abed-284be1654094-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.168006 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.167986    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-metrics-certs\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.202956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.202927    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ed15c1-serving-cert\""
Nov 02 03:20:34.269314 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.269280    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2zl2\" (UniqueName: \"kubernetes.io/projected/60e5f97c-02aa-4d35-abed-284be1654094-kube-api-access-s2zl2\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.269509 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.269329    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.269509 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.269467    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e5f97c-02aa-4d35-abed-284be1654094-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.269509 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.269483    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-metrics-certs\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.269733 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:20:34.269520    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-serving-cert: secret "maas-api-e2e-api-a-ed15c1-serving-cert" not found
Nov 02 03:20:34.269733 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:20:34.269596    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls podName:60e5f97c-02aa-4d35-abed-284be1654094 nodeName:}" failed. No retries permitted until 2026-11-02 03:20:34.769576112 +0000 UTC m=+1978.564924673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls") pod "maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7" (UID: "60e5f97c-02aa-4d35-abed-284be1654094") : secret "maas-api-e2e-api-a-ed15c1-serving-cert" not found
Nov 02 03:20:34.270114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.270088    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e5f97c-02aa-4d35-abed-284be1654094-openshift-service-ca\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.272024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.272003    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-metrics-certs\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.308473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.308436    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:34.329290 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.329261    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2zl2\" (UniqueName: \"kubernetes.io/projected/60e5f97c-02aa-4d35-abed-284be1654094-kube-api-access-s2zl2\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.472870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.472790    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:34.654285 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.654246    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:34.778595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.778559    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:34.781278 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:34.781248    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls\") pod \"maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:35.005061 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:35.005021    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:35.014921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:35.014890    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:35.225070 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:20:35.225026    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:20:35.443015 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:20:35.442973    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e5f97c_02aa_4d35_abed_284be1654094.slice/crio-7e2f69106edcb5ac9e56a29ef324cf033ab2e495c187832071e24def82f20832 WatchSource:0}: Error finding container 7e2f69106edcb5ac9e56a29ef324cf033ab2e495c187832071e24def82f20832: Status 404 returned error can't find the container with id 7e2f69106edcb5ac9e56a29ef324cf033ab2e495c187832071e24def82f20832
Nov 02 03:20:35.443476 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:35.443415    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"]
Nov 02 03:20:35.521639 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:35.521589    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:36.414714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:36.414680    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7" event={"ID":"60e5f97c-02aa-4d35-abed-284be1654094","Type":"ContainerStarted","Data":"2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9"}
Nov 02 03:20:36.415198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:36.414742    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7" event={"ID":"60e5f97c-02aa-4d35-abed-284be1654094","Type":"ContainerStarted","Data":"7e2f69106edcb5ac9e56a29ef324cf033ab2e495c187832071e24def82f20832"}
Nov 02 03:20:36.415198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:36.415132    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:36.574387 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:36.574281    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7" podStartSLOduration=1.740854143 podStartE2EDuration="2.57426828s" totalImagesPullingTime="833.414137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:20:36.5707393 +0000 UTC m=+1980.366087867" watchObservedRunningTime="2026-11-02 03:20:36.57426828 +0000 UTC m=+1980.369616845"
Nov 02 03:20:39.234137 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:39.234104    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:39.746403 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:39.746367    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:39.912539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:39.912510    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:40.079096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:40.079016    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:40.852855 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:40.852822    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:41.029746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:41.029716    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:41.201831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:41.201750    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:42.429250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:42.429224    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:20:45.040686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:45.040653    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:45.203644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:45.203451    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:45.368139 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:45.368048    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:49.880286 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:49.880248    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:52.772884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:52.772845    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:20:54.806762 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:54.806722    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:57.420452 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:57.420417    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:20:59.593689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:20:59.593657    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:02.585330 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:02.585298    4407 generic.go:356] "Generic (PLEG): container finished" podID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerID="3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c" exitCode=0
Nov 02 03:21:02.585921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:02.585382    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" event={"ID":"c42b386b-9a17-4c21-b5f4-7cc0934d358a","Type":"ContainerDied","Data":"3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c"}
Nov 02 03:21:02.585991 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:02.585944    4407 scope.go:122] "RemoveContainer" containerID="3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c"
Nov 02 03:21:02.587527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:02.587494    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:02.587657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:02.587573    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:02.587657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:02.587609    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:03.614034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:03.614001    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" event={"ID":"c42b386b-9a17-4c21-b5f4-7cc0934d358a","Type":"ContainerStarted","Data":"6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483"}
Nov 02 03:21:03.758288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:03.758253    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:21:03.758558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:03.758538    4407 patch_prober.go:29] interesting pod/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 03:21:03.758609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:03.758590    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 03:21:04.320168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:04.320138    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:04.758524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:04.758496    4407 patch_prober.go:29] interesting pod/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 03:21:04.758964 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:04.758548    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 03:21:05.212734 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:21:05.212641    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:21:05.758553 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:05.758513    4407 patch_prober.go:29] interesting pod/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 03:21:05.758987 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:05.758567    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 03:21:06.758011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:06.757977    4407 patch_prober.go:29] interesting pod/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 03:21:06.758228 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:06.758035    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 03:21:07.758348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:07.758310    4407 patch_prober.go:29] interesting pod/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 03:21:07.758794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:07.758376    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 03:21:08.782385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:08.782355    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:21:08.782896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:08.782861    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:21:08.783749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:08.783729    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:21:09.079319 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:09.079217    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:13.907747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:13.907713    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:17.163004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:17.162971    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:21:18.816965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:18.816887    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:23.408469 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.408429    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"]
Nov 02 03:21:23.411951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.411927    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.458234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.458209    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"]
Nov 02 03:21:23.506112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.506071    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b4a230-serving-cert\""
Nov 02 03:21:23.524050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.524016    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-metrics-certs\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.524245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.524104    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245451d8-d8c5-4b7c-8445-3460460cf9fd-openshift-service-ca\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.524245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.524238    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.524394 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.524375    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c46qw\" (UniqueName: \"kubernetes.io/projected/245451d8-d8c5-4b7c-8445-3460460cf9fd-kube-api-access-c46qw\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.626715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.626663    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.626917 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:21:23.626786    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-serving-cert: secret "maas-api-e2e-api-a-b4a230-serving-cert" not found
Nov 02 03:21:23.626917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.626841    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c46qw\" (UniqueName: \"kubernetes.io/projected/245451d8-d8c5-4b7c-8445-3460460cf9fd-kube-api-access-c46qw\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.626917 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:21:23.626858    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls podName:245451d8-d8c5-4b7c-8445-3460460cf9fd nodeName:}" failed. No retries permitted until 2026-11-02 03:21:24.126838163 +0000 UTC m=+2027.922186706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls") pod "maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l" (UID: "245451d8-d8c5-4b7c-8445-3460460cf9fd") : secret "maas-api-e2e-api-a-b4a230-serving-cert" not found
Nov 02 03:21:23.627099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.626979    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-metrics-certs\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.627099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.627061    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245451d8-d8c5-4b7c-8445-3460460cf9fd-openshift-service-ca\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.628026 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.628003    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245451d8-d8c5-4b7c-8445-3460460cf9fd-openshift-service-ca\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.629514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.629494    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-metrics-certs\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.640905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.640872    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:23.699900 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.699817    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c46qw\" (UniqueName: \"kubernetes.io/projected/245451d8-d8c5-4b7c-8445-3460460cf9fd-kube-api-access-c46qw\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:23.847093 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:23.847055    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:24.136557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:24.136518    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:24.139234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:24.139209    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls\") pod \"maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:24.336885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:24.336851    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:24.713711 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:24.713677    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"]
Nov 02 03:21:24.714149 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:21:24.714119    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245451d8_d8c5_4b7c_8445_3460460cf9fd.slice/crio-79527511fd5a5e9a6dfe68baf2a849c41ae8c9fcc98e253d4a5d133861d90478 WatchSource:0}: Error finding container 79527511fd5a5e9a6dfe68baf2a849c41ae8c9fcc98e253d4a5d133861d90478: Status 404 returned error can't find the container with id 79527511fd5a5e9a6dfe68baf2a849c41ae8c9fcc98e253d4a5d133861d90478
Nov 02 03:21:24.754589 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:24.754547    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l" event={"ID":"245451d8-d8c5-4b7c-8445-3460460cf9fd","Type":"ContainerStarted","Data":"79527511fd5a5e9a6dfe68baf2a849c41ae8c9fcc98e253d4a5d133861d90478"}
Nov 02 03:21:25.763006 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:25.762970    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l" event={"ID":"245451d8-d8c5-4b7c-8445-3460460cf9fd","Type":"ContainerStarted","Data":"dd657d4a69ca5a7d61dd1c47996cbe7dda64910be045dbf684e4fc7c4c6f46dd"}
Nov 02 03:21:25.763507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:25.763075    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:25.881952 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:25.881901    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l" podStartSLOduration=2.126280898 podStartE2EDuration="2.881886941s" totalImagesPullingTime="755.606043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:21:25.878916486 +0000 UTC m=+2029.674265050" watchObservedRunningTime="2026-11-02 03:21:25.881886941 +0000 UTC m=+2029.677235506"
Nov 02 03:21:27.009789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:27.009744    4407 patch_prober.go:29] interesting pod/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:21:27.010314 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:27.009800    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:21:27.783183 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:27.783151    4407 generic.go:356] "Generic (PLEG): container finished" podID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerID="88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01" exitCode=0
Nov 02 03:21:27.783402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:27.783206    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" event={"ID":"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b","Type":"ContainerDied","Data":"88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01"}
Nov 02 03:21:27.783654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:27.783606    4407 scope.go:122] "RemoveContainer" containerID="88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01"
Nov 02 03:21:27.785304 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:27.785268    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:27.785377 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:27.785343    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:27.785413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:27.785383    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:28.556317 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:28.556289    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:28.814001 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:28.813778    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" event={"ID":"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b","Type":"ContainerStarted","Data":"029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608"}
Nov 02 03:21:28.906436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:28.906400    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:28.947172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:28.947140    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:21:28.947353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:28.947327    4407 patch_prober.go:29] interesting pod/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 03:21:28.947413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:28.947372    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 03:21:29.262878 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.262828    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:29.730084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.729973    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"]
Nov 02 03:21:29.735202 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.735178    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.884704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.849608    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-podinfo\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.885062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.885028    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-token\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.885244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.885225    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-credential-socket\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.885858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.885834    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-data\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.886040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.886023    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-socket\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.886212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.886198    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-certs\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.886385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.886372    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-envoy\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.886516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.886504    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istiod-ca-cert\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.886666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.886650    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdzmg\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-kube-api-access-gdzmg\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:29.891093 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.891067    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d6544b-openshift-default-dockercfg-crlrt\""
Nov 02 03:21:29.900892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.900861    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:29.921507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.921458    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"]
Nov 02 03:21:29.950315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.950284    4407 patch_prober.go:29] interesting pod/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 03:21:29.950589 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:29.950558    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 03:21:30.000497 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.000458    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-certs\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.000850 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.000820    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-envoy\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.001049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.001029    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istiod-ca-cert\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.001202 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.001183    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdzmg\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-kube-api-access-gdzmg\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.001542 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.001517    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-podinfo\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.001763 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.001743    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-token\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.001942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.001924    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-credential-socket\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.002068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.002051    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-data\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.002220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.002203    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-socket\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.002898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.002876    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-socket\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.004540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.004514    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-certs\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.005353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.005332    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-data\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.007397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.007309    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istiod-ca-cert\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.008257 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.008205    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-credential-socket\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.016507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.016481    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-envoy\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.017840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.017816    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-podinfo\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.071692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.071581    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:30.077391 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.077358    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdzmg\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-kube-api-access-gdzmg\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.121659 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.112343    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-token\") pod \"e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") " pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.349649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.349551    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:30.807995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.807966    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"]
Nov 02 03:21:30.817684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.817616    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:30.817799 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.817775    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:30.817877 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.817860    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:21:30.914616 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.914581    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" event={"ID":"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9","Type":"ContainerStarted","Data":"87ad3e473ab9c8d65dfd367ff7f5d2f999f160a48d74520a3ca67fdc561d67a3"}
Nov 02 03:21:30.946722 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.946681    4407 patch_prober.go:29] interesting pod/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 03:21:30.946906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:30.946759    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 03:21:31.806318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:31.806223    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:21:31.923896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:31.923854    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" event={"ID":"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9","Type":"ContainerStarted","Data":"6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709"}
Nov 02 03:21:31.947461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:31.947430    4407 patch_prober.go:29] interesting pod/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 03:21:31.947735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:31.947701    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 03:21:32.047658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:32.047592    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" podStartSLOduration=3.047579112 podStartE2EDuration="3.047579112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:21:32.044322471 +0000 UTC m=+2035.839671038" watchObservedRunningTime="2026-11-02 03:21:32.047579112 +0000 UTC m=+2035.842927676"
Nov 02 03:21:32.352217 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:32.352191    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:32.355411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:32.355387    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:32.930954 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:32.930910    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:32.931936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:32.931912    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:21:32.946647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:32.946608    4407 patch_prober.go:29] interesting pod/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 02 03:21:32.946796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:32.946673    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 02 03:21:33.349650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:33.349603    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:34.003022 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:34.002992    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:34.165260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:34.165139    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:34.331762 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:34.328351    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:34.424235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:34.424201    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:21:34.424922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:34.424896    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:21:34.426050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:34.426027    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:21:34.804410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:34.804374    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:35.190238 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:21:35.190143    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:21:35.375072 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:35.375008    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:35.656911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:35.656867    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:35.897925 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:35.897825    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:36.084457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:36.084420    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:36.257801 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:36.257766    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:36.599762 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:36.599723    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:39.129876 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:39.129844    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:39.300125 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:39.300089    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:39.469143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:39.468989    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:39.650106 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:39.650071    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:39.822399 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:39.822365    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:42.977375 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:42.977339    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:43.150471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:43.150437    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:43.313913 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:43.313879    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:44.676918 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:44.676888    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:45.158525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:45.158493    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:45.351927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:45.351899    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:47.655333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:47.655297    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:47.848149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:47.848073    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:50.019187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:50.019156    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:51.506014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:51.505977    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:51.662445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:51.662413    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:55.074334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:55.074288    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:21:59.959289 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:21:59.959258    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:01.501436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:01.501402    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:22:04.905367 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:04.905336    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:05.189116 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:22:05.189027    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:22:09.781899 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:09.781870    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:14.873821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:14.873792    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:16.130707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:16.130663    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:16.832163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:16.832122    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:17.328468 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.328433    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"]
Nov 02 03:22:17.334809 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.334789    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.400994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.400961    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a27da5-openshift-default-dockercfg-stf5m\""
Nov 02 03:22:17.423592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.423551    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-token\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.449649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.439463    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-credential-socket\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.449649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.439502    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-podinfo\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.449649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.439576    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6fxw\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-kube-api-access-j6fxw\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.449649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.439601    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e61c64-563d-4c06-b771-89b52ff8191b-istiod-ca-cert\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.449649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.439758    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-envoy\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.449649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.439830    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-data\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.449649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.439869    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-socket\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.449649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.439936    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-certs\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.460334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.460305    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"]
Nov 02 03:22:17.514151 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.514020    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:17.541204 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541166    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6fxw\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-kube-api-access-j6fxw\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.541413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541223    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e61c64-563d-4c06-b771-89b52ff8191b-istiod-ca-cert\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.541413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541323    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-envoy\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.541413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541382    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-data\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.541603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541428    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-socket\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.541603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541492    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-certs\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.541603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541574    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-token\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.541807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541655    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-credential-socket\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.541807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.541682    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-podinfo\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.542460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.542174    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-socket\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.542460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.542414    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-credential-socket\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.542705 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.542678    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-certs\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.542901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.542877    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-data\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.543306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.543284    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e61c64-563d-4c06-b771-89b52ff8191b-istiod-ca-cert\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.544696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.544671    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-envoy\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.546194 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.545478    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-podinfo\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.600683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.600508    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-token\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.601020 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.600963    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6fxw\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-kube-api-access-j6fxw\") pod \"e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") " pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.652250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.652216    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:17.745602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:17.745472    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:18.093196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:18.093160    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"]
Nov 02 03:22:18.196721 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:18.196667    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:22:18.197004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:18.196983    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:22:18.197157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:18.197141    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:22:18.306758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:18.306596    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" event={"ID":"e6e61c64-563d-4c06-b771-89b52ff8191b","Type":"ContainerStarted","Data":"04885c1ebd0f4c2fd74694b156918ac41b28c815fb735d7fe20ae67505ddf17c"}
Nov 02 03:22:19.324330 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:19.324290    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" event={"ID":"e6e61c64-563d-4c06-b771-89b52ff8191b","Type":"ContainerStarted","Data":"15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a"}
Nov 02 03:22:19.544691 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:19.544616    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" podStartSLOduration=2.544601922 podStartE2EDuration="2.544601922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:22:19.493149127 +0000 UTC m=+2083.288497693" watchObservedRunningTime="2026-11-02 03:22:19.544601922 +0000 UTC m=+2083.339950486"
Nov 02 03:22:19.654733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:19.652941    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:19.659369 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:19.659342    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:20.331299 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:20.331268    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:20.332187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:20.332169    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:22:20.514435 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:20.514408    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:20.665843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:20.665747    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:20.817936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:20.817903    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:22.399291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:22.399264    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:22.572536 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:22.572467    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:22.740543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:22.740461    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:25.639491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:25.639461    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:25.810061 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:25.810025    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:26.003848 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:26.003813    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:30.376918 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:30.376884    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:22:31.111560 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.111527    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"]
Nov 02 03:22:31.115639 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.115605    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.206994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.206951    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-metrics-certs\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.207188 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.207123    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.207342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.207322    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxgjj\" (UniqueName: \"kubernetes.io/projected/667af376-6718-4ce6-8acf-2b3399c5282f-kube-api-access-rxgjj\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.207487 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.207464    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667af376-6718-4ce6-8acf-2b3399c5282f-openshift-service-ca\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.219122 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.219100    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c9d1fc-serving-cert\""
Nov 02 03:22:31.222702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.222678    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"]
Nov 02 03:22:31.308706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.308675    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-metrics-certs\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.308877 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.308758    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.308960 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:22:31.308888    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-serving-cert: secret "maas-api-e2e-api-a-c9d1fc-serving-cert" not found
Nov 02 03:22:31.308960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.308924    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxgjj\" (UniqueName: \"kubernetes.io/projected/667af376-6718-4ce6-8acf-2b3399c5282f-kube-api-access-rxgjj\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.309072 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:22:31.308962    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls podName:667af376-6718-4ce6-8acf-2b3399c5282f nodeName:}" failed. No retries permitted until 2026-11-02 03:22:31.808941115 +0000 UTC m=+2095.604289678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls") pod "maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h" (UID: "667af376-6718-4ce6-8acf-2b3399c5282f") : secret "maas-api-e2e-api-a-c9d1fc-serving-cert" not found
Nov 02 03:22:31.309131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.309081    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667af376-6718-4ce6-8acf-2b3399c5282f-openshift-service-ca\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.309714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.309695    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667af376-6718-4ce6-8acf-2b3399c5282f-openshift-service-ca\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.311329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.311301    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-metrics-certs\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.330769 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.330746    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:31.370018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.369904    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxgjj\" (UniqueName: \"kubernetes.io/projected/667af376-6718-4ce6-8acf-2b3399c5282f-kube-api-access-rxgjj\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.514254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.514219    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:31.716536 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.716446    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:31.818121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.818091    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:31.820708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:31.820688    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls\") pod \"maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:32.029807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:32.029773    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:32.408566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:32.408540    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"]
Nov 02 03:22:32.409706 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:22:32.409677    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667af376_6718_4ce6_8acf_2b3399c5282f.slice/crio-91dee6e884612c3db2f79281cd285d723d4635fe712889c8bd2a02ccdda09d3f WatchSource:0}: Error finding container 91dee6e884612c3db2f79281cd285d723d4635fe712889c8bd2a02ccdda09d3f: Status 404 returned error can't find the container with id 91dee6e884612c3db2f79281cd285d723d4635fe712889c8bd2a02ccdda09d3f
Nov 02 03:22:32.424278 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:32.424250    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h" event={"ID":"667af376-6718-4ce6-8acf-2b3399c5282f","Type":"ContainerStarted","Data":"91dee6e884612c3db2f79281cd285d723d4635fe712889c8bd2a02ccdda09d3f"}
Nov 02 03:22:33.436216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:33.436174    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h" event={"ID":"667af376-6718-4ce6-8acf-2b3399c5282f","Type":"ContainerStarted","Data":"5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a"}
Nov 02 03:22:33.436764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:33.436407    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:33.555834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:33.555788    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h" podStartSLOduration=1.761543453 podStartE2EDuration="2.555773273s" totalImagesPullingTime="794.22982ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:22:33.553837652 +0000 UTC m=+2097.349186217" watchObservedRunningTime="2026-11-02 03:22:33.555773273 +0000 UTC m=+2097.351121837"
Nov 02 03:22:35.189996 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:22:35.189954    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:22:36.489424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.489391    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"]
Nov 02 03:22:36.493894 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.493873    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.550111 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.550079    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"]
Nov 02 03:22:36.594505 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.594475    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.594695 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.594650    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vntfs\" (UniqueName: \"kubernetes.io/projected/de6da90e-8860-48f3-beea-6b6819b6481d-kube-api-access-vntfs\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.594825 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.594799    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-metrics-certs\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.594955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.594890    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6da90e-8860-48f3-beea-6b6819b6481d-openshift-service-ca\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.595896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.595872    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-018cfa-serving-cert\""
Nov 02 03:22:36.696200 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.696153    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vntfs\" (UniqueName: \"kubernetes.io/projected/de6da90e-8860-48f3-beea-6b6819b6481d-kube-api-access-vntfs\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.696366 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.696295    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-metrics-certs\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.696408 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.696382    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6da90e-8860-48f3-beea-6b6819b6481d-openshift-service-ca\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.696514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.696498    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.696683 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:22:36.696667    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-serving-cert: secret "maas-api-e2e-api-b-018cfa-serving-cert" not found
Nov 02 03:22:36.696829 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:22:36.696722    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls podName:de6da90e-8860-48f3-beea-6b6819b6481d nodeName:}" failed. No retries permitted until 2026-11-02 03:22:37.196708757 +0000 UTC m=+2100.992057300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls") pod "maas-api-e2e-api-b-018cfa-5bb997764b-9p77k" (UID: "de6da90e-8860-48f3-beea-6b6819b6481d") : secret "maas-api-e2e-api-b-018cfa-serving-cert" not found
Nov 02 03:22:36.697345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.697321    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6da90e-8860-48f3-beea-6b6819b6481d-openshift-service-ca\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.699120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.699095    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-metrics-certs\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.754237 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.754211    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:36.756751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.756730    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vntfs\" (UniqueName: \"kubernetes.io/projected/de6da90e-8860-48f3-beea-6b6819b6481d-kube-api-access-vntfs\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:36.964903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:36.964869    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:37.119246 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:37.119158    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:37.206039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:37.206010    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:37.208512 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:37.208490    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls\") pod \"maas-api-e2e-api-b-018cfa-5bb997764b-9p77k\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:37.408002 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:37.407908    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:37.783348 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:22:37.783311    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6da90e_8860_48f3_beea_6b6819b6481d.slice/crio-1b167635d299523c7d7d489d518f50ef60c333548b02d523add8b43455d2918b WatchSource:0}: Error finding container 1b167635d299523c7d7d489d518f50ef60c333548b02d523add8b43455d2918b: Status 404 returned error can't find the container with id 1b167635d299523c7d7d489d518f50ef60c333548b02d523add8b43455d2918b
Nov 02 03:22:37.783739 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:37.783358    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"]
Nov 02 03:22:38.515265 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:38.515230    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k" event={"ID":"de6da90e-8860-48f3-beea-6b6819b6481d","Type":"ContainerStarted","Data":"1b167635d299523c7d7d489d518f50ef60c333548b02d523add8b43455d2918b"}
Nov 02 03:22:39.486785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:39.486758    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:22:39.522701 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:39.522665    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k" event={"ID":"de6da90e-8860-48f3-beea-6b6819b6481d","Type":"ContainerStarted","Data":"0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631"}
Nov 02 03:22:39.522896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:39.522731    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:39.717208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:39.717154    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k" podStartSLOduration=2.925517128 podStartE2EDuration="3.717139526s" totalImagesPullingTime="791.622398ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:22:39.716263333 +0000 UTC m=+2103.511611899" watchObservedRunningTime="2026-11-02 03:22:39.717139526 +0000 UTC m=+2103.512488091"
Nov 02 03:22:40.044476 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:40.044443    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:42.124852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:42.124812    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:45.536287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:45.536256    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:22:46.969103 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:46.969069    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:51.805944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:51.804231    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:22:56.666560 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:22:56.666529    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:00.680255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.680218    4407 generic.go:356] "Generic (PLEG): container finished" podID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerID="6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709" exitCode=0
Nov 02 03:23:00.680731 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.680290    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" event={"ID":"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9","Type":"ContainerDied","Data":"6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709"}
Nov 02 03:23:00.680816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.680799    4407 scope.go:122] "RemoveContainer" containerID="6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709"
Nov 02 03:23:00.682541 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.682509    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:23:00.682614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.682581    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:23:00.682689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.682655    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:23:00.790390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.790355    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"]
Nov 02 03:23:00.790657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.790610    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v" podUID="35c59f44-c65d-41d7-932b-70d82d0dc930" containerName="maas-api" containerID="cri-o://1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d" gracePeriod=30
Nov 02 03:23:00.889788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:00.889757    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:01.128346 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.128317    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:23:01.144850 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.144817    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35c59f44-c65d-41d7-932b-70d82d0dc930-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c59f44-c65d-41d7-932b-70d82d0dc930-openshift-service-ca\") pod \"35c59f44-c65d-41d7-932b-70d82d0dc930\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") "
Nov 02 03:23:01.145191 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.145169    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-metrics-certs\") pod \"35c59f44-c65d-41d7-932b-70d82d0dc930\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") "
Nov 02 03:23:01.145302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.145215    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls\") pod \"35c59f44-c65d-41d7-932b-70d82d0dc930\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") "
Nov 02 03:23:01.145302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.145245    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35c59f44-c65d-41d7-932b-70d82d0dc930-kube-api-access-fg9xc\" (UniqueName: \"kubernetes.io/projected/35c59f44-c65d-41d7-932b-70d82d0dc930-kube-api-access-fg9xc\") pod \"35c59f44-c65d-41d7-932b-70d82d0dc930\" (UID: \"35c59f44-c65d-41d7-932b-70d82d0dc930\") "
Nov 02 03:23:01.145302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.145171    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c59f44-c65d-41d7-932b-70d82d0dc930-openshift-service-ca" pod "35c59f44-c65d-41d7-932b-70d82d0dc930" (UID: "35c59f44-c65d-41d7-932b-70d82d0dc930"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:23:01.146079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.146055    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c59f44-c65d-41d7-932b-70d82d0dc930-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:01.147945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.147922    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c59f44-c65d-41d7-932b-70d82d0dc930-kube-api-access-fg9xc" pod "35c59f44-c65d-41d7-932b-70d82d0dc930" (UID: "35c59f44-c65d-41d7-932b-70d82d0dc930"). InnerVolumeSpecName "kube-api-access-fg9xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:01.148072 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.148012    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls" pod "35c59f44-c65d-41d7-932b-70d82d0dc930" (UID: "35c59f44-c65d-41d7-932b-70d82d0dc930"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:23:01.148985 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.148964    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-metrics-certs" pod "35c59f44-c65d-41d7-932b-70d82d0dc930" (UID: "35c59f44-c65d-41d7-932b-70d82d0dc930"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:23:01.253685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.247567    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:01.253685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.247606    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c59f44-c65d-41d7-932b-70d82d0dc930-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:01.253685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.247642    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg9xc\" (UniqueName: \"kubernetes.io/projected/35c59f44-c65d-41d7-932b-70d82d0dc930-kube-api-access-fg9xc\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:01.713073 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.712976    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" event={"ID":"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9","Type":"ContainerStarted","Data":"612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75"}
Nov 02 03:23:01.714469 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.714442    4407 generic.go:356] "Generic (PLEG): container finished" podID="35c59f44-c65d-41d7-932b-70d82d0dc930" containerID="1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d" exitCode=0
Nov 02 03:23:01.714590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.714517    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"
Nov 02 03:23:01.714658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.714521    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v" event={"ID":"35c59f44-c65d-41d7-932b-70d82d0dc930","Type":"ContainerDied","Data":"1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d"}
Nov 02 03:23:01.714709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.714668    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v" event={"ID":"35c59f44-c65d-41d7-932b-70d82d0dc930","Type":"ContainerDied","Data":"e2bbbb67ec6262b6e73272ab4cf875084618e6e12b4be843456ad2c82fb1bc9a"}
Nov 02 03:23:01.714709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.714693    4407 scope.go:122] "RemoveContainer" containerID="1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d"
Nov 02 03:23:01.725438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.725418    4407 scope.go:122] "RemoveContainer" containerID="1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d"
Nov 02 03:23:01.725748 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:23:01.725728    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d\": container with ID starting with 1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d not found: ID does not exist" containerID="1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d"
Nov 02 03:23:01.725813 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.725756    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d"} err="failed to get container status \"1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d\": rpc error: code = NotFound desc = could not find container \"1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d\": container with ID starting with 1531da3af68bf3171eeba82b008b1fcc5f1f2ecbc608017c3204f52129ab4c1d not found: ID does not exist"
Nov 02 03:23:01.977062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.975775    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"]
Nov 02 03:23:01.989276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:01.989244    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d5dcca72-565b874c9b-r628v"]
Nov 02 03:23:02.351293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:02.351261    4407 patch_prober.go:29] interesting pod/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 03:23:02.351464 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:02.351331    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 03:23:02.353004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:02.352984    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35c59f44-c65d-41d7-932b-70d82d0dc930" path="/var/lib/kubelet/pods/35c59f44-c65d-41d7-932b-70d82d0dc930/volumes"
Nov 02 03:23:02.353292 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:02.353280    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:23:02.553892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:02.553865    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:02.765322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:02.765288    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:02.926717 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:02.926678    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:02.930988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:02.930967    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:23:03.350711 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:03.350682    4407 patch_prober.go:29] interesting pod/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 03:23:03.350897 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:03.350730    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 03:23:04.349868 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:04.349833    4407 patch_prober.go:29] interesting pod/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 03:23:04.350352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:04.349892    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 03:23:05.214108 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:23:05.214071    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:23:05.349984 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:05.349952    4407 patch_prober.go:29] interesting pod/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 03:23:05.350367 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:05.350013    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 03:23:06.352202 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:06.352166    4407 patch_prober.go:29] interesting pod/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 03:23:06.352612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:06.352243    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 03:23:06.831163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:06.831134    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:07.355131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:07.355102    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:23:07.356251 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:07.356233    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:23:08.042827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:08.042793    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:13.106376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:13.106340    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:13.267749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:13.267720    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:13.499817 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:13.499789    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:15.622826 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:15.622779    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:15.798502 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:15.798466    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:17.373143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:17.373111    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:23:18.735027 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:18.734992    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:18.909098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:18.909065    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:23.480942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.480907    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"]
Nov 02 03:23:23.481406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.481124    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy" containerID="cri-o://aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6" gracePeriod=30
Nov 02 03:23:23.531692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.531659    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"]
Nov 02 03:23:23.531929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.531905    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" containerID="cri-o://5cce861eb307379ee7ad63bb176d097073a74d471a21e8592819315a0188921c" gracePeriod=30
Nov 02 03:23:23.535210 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.535183    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"]
Nov 02 03:23:23.535442 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.535414    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" containerID="cri-o://18cbc71793167f6dac55ad09632b1bd8881d370ea35b2babd9626a28390d223d" gracePeriod=30
Nov 02 03:23:23.584103 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.584070    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"]
Nov 02 03:23:23.584471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.584305    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy" containerID="cri-o://815ce265a1eb4aa57a1e02e635b0cbac12629a2bb2bba2af7b23eeb295a4de80" gracePeriod=30
Nov 02 03:23:23.588930 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.588907    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"]
Nov 02 03:23:23.589610 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.589591    4407 cpu_manager.go:414] "containerMap: removing container" podUID="35c59f44-c65d-41d7-932b-70d82d0dc930" containerName="maas-api"
Nov 02 03:23:23.589742 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.589611    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35c59f44-c65d-41d7-932b-70d82d0dc930" containerName="maas-api"
Nov 02 03:23:23.589853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.589839    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="35c59f44-c65d-41d7-932b-70d82d0dc930" containerName="maas-api"
Nov 02 03:23:23.594790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.594774    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.648210 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.648089    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a27da5-serving-cert\""
Nov 02 03:23:23.655094 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.655067    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"]
Nov 02 03:23:23.736254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.736160    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dc888ac-f7c8-483c-bbda-75ec89265b77-openshift-service-ca\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.736254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.736237    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krblb\" (UniqueName: \"kubernetes.io/projected/1dc888ac-f7c8-483c-bbda-75ec89265b77-kube-api-access-krblb\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.736445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.736326    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-metrics-certs\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.736552 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.736533    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.838427 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.838399    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.838592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.838494    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dc888ac-f7c8-483c-bbda-75ec89265b77-openshift-service-ca\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.838592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.838544    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krblb\" (UniqueName: \"kubernetes.io/projected/1dc888ac-f7c8-483c-bbda-75ec89265b77-kube-api-access-krblb\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.838592 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:23:23.838556    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-serving-cert: secret "maas-api-e2e-api-b-a27da5-serving-cert" not found
Nov 02 03:23:23.838592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.838568    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-metrics-certs\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.838798 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:23:23.838666    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls podName:1dc888ac-f7c8-483c-bbda-75ec89265b77 nodeName:}" failed. No retries permitted until 2026-11-02 03:23:24.338611048 +0000 UTC m=+2148.133959593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls") pod "maas-api-e2e-api-b-a27da5-66946cdd4c-pm567" (UID: "1dc888ac-f7c8-483c-bbda-75ec89265b77") : secret "maas-api-e2e-api-b-a27da5-serving-cert" not found
Nov 02 03:23:23.839181 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.839162    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dc888ac-f7c8-483c-bbda-75ec89265b77-openshift-service-ca\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.841113 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.841089    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-metrics-certs\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:23.879609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.879579    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:23.898890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:23.898862    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krblb\" (UniqueName: \"kubernetes.io/projected/1dc888ac-f7c8-483c-bbda-75ec89265b77-kube-api-access-krblb\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:24.090021 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:24.089978    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:24.350385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:24.350174    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:24.353494 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:24.353461    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls\") pod \"maas-api-e2e-api-b-a27da5-66946cdd4c-pm567\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:24.510572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:24.510539    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:24.888516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:24.888493    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"]
Nov 02 03:23:24.889110 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:23:24.889084    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc888ac_f7c8_483c_bbda_75ec89265b77.slice/crio-28f6a22e3811e704b20ba75aaad4a5db54a540a12fa1f2a22ee0c9b260a59303 WatchSource:0}: Error finding container 28f6a22e3811e704b20ba75aaad4a5db54a540a12fa1f2a22ee0c9b260a59303: Status 404 returned error can't find the container with id 28f6a22e3811e704b20ba75aaad4a5db54a540a12fa1f2a22ee0c9b260a59303
Nov 02 03:23:25.105901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:25.105870    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"]
Nov 02 03:23:25.106125 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:25.106090    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" containerID="cri-o://7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f" gracePeriod=30
Nov 02 03:23:25.157917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:25.157831    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"]
Nov 02 03:23:25.158163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:25.158109    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy" containerID="cri-o://21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1" gracePeriod=30
Nov 02 03:23:25.883953 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:25.883918    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567" event={"ID":"1dc888ac-f7c8-483c-bbda-75ec89265b77","Type":"ContainerStarted","Data":"28f6a22e3811e704b20ba75aaad4a5db54a540a12fa1f2a22ee0c9b260a59303"}
Nov 02 03:23:26.903377 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:26.897131    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567" event={"ID":"1dc888ac-f7c8-483c-bbda-75ec89265b77","Type":"ContainerStarted","Data":"d9a7bbffd359076b8659581daa08eb90c16167d1de1e6f240d94b9c79f1c62c9"}
Nov 02 03:23:26.903377 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:26.897865    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:26.926114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:26.926080    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:23:26.926279 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:26.926148    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:23:27.016719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:27.016658    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567" podStartSLOduration=2.498075209 podStartE2EDuration="4.016638253s" totalImagesPullingTime="1.518563044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:23:27.013353554 +0000 UTC m=+2150.808702130" watchObservedRunningTime="2026-11-02 03:23:27.016638253 +0000 UTC m=+2150.811986817"
Nov 02 03:23:28.879232 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.879201    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:23:28.914457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.914418    4407 generic.go:356] "Generic (PLEG): container finished" podID="554b9b58-87b5-443e-908a-56c80a422a4e" containerID="815ce265a1eb4aa57a1e02e635b0cbac12629a2bb2bba2af7b23eeb295a4de80" exitCode=0
Nov 02 03:23:28.914602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.914490    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" event={"ID":"554b9b58-87b5-443e-908a-56c80a422a4e","Type":"ContainerDied","Data":"815ce265a1eb4aa57a1e02e635b0cbac12629a2bb2bba2af7b23eeb295a4de80"}
Nov 02 03:23:28.916435 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.916411    4407 generic.go:356] "Generic (PLEG): container finished" podID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerID="18cbc71793167f6dac55ad09632b1bd8881d370ea35b2babd9626a28390d223d" exitCode=0
Nov 02 03:23:28.916558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.916484    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" event={"ID":"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf","Type":"ContainerDied","Data":"18cbc71793167f6dac55ad09632b1bd8881d370ea35b2babd9626a28390d223d"}
Nov 02 03:23:28.918422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.918396    4407 generic.go:356] "Generic (PLEG): container finished" podID="3d358326-31d3-477c-85cc-5e5069e03e16" containerID="aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6" exitCode=0
Nov 02 03:23:28.918537 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.918493    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" event={"ID":"3d358326-31d3-477c-85cc-5e5069e03e16","Type":"ContainerDied","Data":"aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6"}
Nov 02 03:23:28.918537 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.918533    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c" event={"ID":"3d358326-31d3-477c-85cc-5e5069e03e16","Type":"ContainerDied","Data":"5323e7fc7cd8d28b76c167ad1907f0290f5ea64e72b3ae05a53980e4a8e74ffe"}
Nov 02 03:23:28.918713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.918564    4407 scope.go:122] "RemoveContainer" containerID="aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6"
Nov 02 03:23:28.918944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.918829    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"
Nov 02 03:23:28.920770 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.920748    4407 generic.go:356] "Generic (PLEG): container finished" podID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerID="5cce861eb307379ee7ad63bb176d097073a74d471a21e8592819315a0188921c" exitCode=0
Nov 02 03:23:28.920877 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.920812    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" event={"ID":"587a7c84-880a-4726-bc56-bf1e47c2a98c","Type":"ContainerDied","Data":"5cce861eb307379ee7ad63bb176d097073a74d471a21e8592819315a0188921c"}
Nov 02 03:23:28.933973 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.933949    4407 scope.go:122] "RemoveContainer" containerID="aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6"
Nov 02 03:23:28.934554 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:23:28.934530    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6\": container with ID starting with aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6 not found: ID does not exist" containerID="aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6"
Nov 02 03:23:28.934726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.934699    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6"} err="failed to get container status \"aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6\": rpc error: code = NotFound desc = could not find container \"aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6\": container with ID starting with aacbb48ff43fa62c2616fe6d59c8ace3a3a63d99c8063a3c20a9eead15ac73e6 not found: ID does not exist"
Nov 02 03:23:28.937002 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:28.936984    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:23:29.005297 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.005275    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:23:29.009399 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.009368    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:23:29.034854 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.034830    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-credential-socket\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.034983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.034874    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-certs\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.034983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.034919    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istiod-ca-cert\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.034983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.034947    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-socket\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.034983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.034977    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-podinfo\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.035206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.034999    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d358326-31d3-477c-85cc-5e5069e03e16-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d358326-31d3-477c-85cc-5e5069e03e16-istiod-ca-cert\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.035206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035031    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-token\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-token\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.035206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035066    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-kube-api-access-p9dp5\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-kube-api-access-p9dp5\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.035206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035126    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-data\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.035206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035127    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-credential-socket" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.035206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035152    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-envoy\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035221    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-kube-api-access-qsttz\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-kube-api-access-qsttz\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035233    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-certs" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035247    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-istio-token\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-istio-token\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035277    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d358326-31d3-477c-85cc-5e5069e03e16-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d358326-31d3-477c-85cc-5e5069e03e16-istio-podinfo\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035333    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istiod-ca-cert" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035354    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-socket\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035479    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-certs\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035501    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-data" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.035516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035510    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-credential-socket\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.036218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035583    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-data\") pod \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\" (UID: \"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf\") "
Nov 02 03:23:29.036218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035697    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-envoy\") pod \"3d358326-31d3-477c-85cc-5e5069e03e16\" (UID: \"3d358326-31d3-477c-85cc-5e5069e03e16\") "
Nov 02 03:23:29.036218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.035713    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-socket" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.036218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036021    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d358326-31d3-477c-85cc-5e5069e03e16-istiod-ca-cert" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:23:29.036443 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036349    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-credential-socket" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.036558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036528    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-socket" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.036773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036723    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-certs" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.036939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036789    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.036939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036814    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.036939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036830    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.036939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036845    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.036939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036861    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d358326-31d3-477c-85cc-5e5069e03e16-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.036939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036876    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.036939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036881    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-data" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.036939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.036890    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.038221 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.038192    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-envoy" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.038332 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.038192    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-envoy" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.038595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.038558    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-token" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:29.038706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.038678    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-istio-token" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:29.039028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.039003    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-kube-api-access-p9dp5" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "kube-api-access-p9dp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:29.039328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.039303    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-podinfo" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:23:29.039572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.039554    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d358326-31d3-477c-85cc-5e5069e03e16-istio-podinfo" pod "3d358326-31d3-477c-85cc-5e5069e03e16" (UID: "3d358326-31d3-477c-85cc-5e5069e03e16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:23:29.039955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.039935    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-kube-api-access-qsttz" pod "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" (UID: "75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf"). InnerVolumeSpecName "kube-api-access-qsttz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:29.137977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.137888    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-socket\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.137977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.137961    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-envoy\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.138209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138008    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-kube-api-access-45kqk\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-kube-api-access-45kqk\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.138209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138045    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-data\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.138209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138074    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-socket\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.138209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138109    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-credential-socket\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.138209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138149    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-socket" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.138209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138172    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/554b9b58-87b5-443e-908a-56c80a422a4e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b9b58-87b5-443e-908a-56c80a422a4e-istiod-ca-cert\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.138467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138233    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-certs\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.138467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138286    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-podinfo\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.138467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138314    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-data" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.138467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138323    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-token\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-token\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.138467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138358    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-credential-socket\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.138467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138420    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-data\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.138756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138473    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-certs\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.138756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138598    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-istio-token\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-istio-token\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.138756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138666    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/587a7c84-880a-4726-bc56-bf1e47c2a98c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/587a7c84-880a-4726-bc56-bf1e47c2a98c-istiod-ca-cert\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.138756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138697    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-envoy\") pod \"587a7c84-880a-4726-bc56-bf1e47c2a98c\" (UID: \"587a7c84-880a-4726-bc56-bf1e47c2a98c\") "
Nov 02 03:23:29.138756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138727    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-kube-api-access-9t9p8\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-kube-api-access-9t9p8\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.139174 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.138797    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/554b9b58-87b5-443e-908a-56c80a422a4e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b9b58-87b5-443e-908a-56c80a422a4e-istio-podinfo\") pod \"554b9b58-87b5-443e-908a-56c80a422a4e\" (UID: \"554b9b58-87b5-443e-908a-56c80a422a4e\") "
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139459    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-data" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139607    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/587a7c84-880a-4726-bc56-bf1e47c2a98c-istiod-ca-cert" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139654    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139678    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139694    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139708    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d358326-31d3-477c-85cc-5e5069e03e16-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139722    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139735    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139750    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9dp5\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-kube-api-access-p9dp5\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139798    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139813    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139827    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsttz\" (UniqueName: \"kubernetes.io/projected/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-kube-api-access-qsttz\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139842    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d358326-31d3-477c-85cc-5e5069e03e16-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139856    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d358326-31d3-477c-85cc-5e5069e03e16-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139870    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139883    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.140141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.139916    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-certs" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.140982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.140732    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-credential-socket" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.141245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.141025    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/554b9b58-87b5-443e-908a-56c80a422a4e-istiod-ca-cert" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:23:29.141590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.141400    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-certs" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.141590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.141512    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-kube-api-access-45kqk" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "kube-api-access-45kqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:29.141590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.141566    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-envoy" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.142859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.142818    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-istio-token" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:29.143677 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.143611    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-token" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:29.144004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.143958    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-kube-api-access-9t9p8" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "kube-api-access-9t9p8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:29.144464 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.144412    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/554b9b58-87b5-443e-908a-56c80a422a4e-istio-podinfo" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:23:29.144727 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.144687    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-socket" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.144930 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.144904    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-credential-socket" pod "554b9b58-87b5-443e-908a-56c80a422a4e" (UID: "554b9b58-87b5-443e-908a-56c80a422a4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.145798 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.145778    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-podinfo" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:23:29.145975 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.145952    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-envoy" pod "587a7c84-880a-4726-bc56-bf1e47c2a98c" (UID: "587a7c84-880a-4726-bc56-bf1e47c2a98c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:29.240938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.240907    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.240938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.240938    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.240951    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.240963    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.240976    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.240987    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.240999    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/587a7c84-880a-4726-bc56-bf1e47c2a98c-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.241010    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/587a7c84-880a-4726-bc56-bf1e47c2a98c-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.241024    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t9p8\" (UniqueName: \"kubernetes.io/projected/554b9b58-87b5-443e-908a-56c80a422a4e-kube-api-access-9t9p8\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.241038    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b9b58-87b5-443e-908a-56c80a422a4e-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.241050    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.241063    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45kqk\" (UniqueName: \"kubernetes.io/projected/587a7c84-880a-4726-bc56-bf1e47c2a98c-kube-api-access-45kqk\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.241078    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.241090    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b9b58-87b5-443e-908a-56c80a422a4e-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.241149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.241104    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b9b58-87b5-443e-908a-56c80a422a4e-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:29.287350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.287317    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:29.579992 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.579959    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"]
Nov 02 03:23:29.583230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.583206    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-q4d4c"]
Nov 02 03:23:29.925762 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.925681    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:23:29.925762 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.925735    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:23:29.929242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.929216    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2" event={"ID":"75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf","Type":"ContainerDied","Data":"79158fde7e56f55586a2838438c3798dadabd2b9cc2f5ab154be5fdb78e3c8d0"}
Nov 02 03:23:29.929378 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.929227    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"
Nov 02 03:23:29.929378 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.929277    4407 scope.go:122] "RemoveContainer" containerID="18cbc71793167f6dac55ad09632b1bd8881d370ea35b2babd9626a28390d223d"
Nov 02 03:23:29.931941 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.931922    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"
Nov 02 03:23:29.932100 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.931962    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" event={"ID":"587a7c84-880a-4726-bc56-bf1e47c2a98c","Type":"ContainerDied","Data":"5994f0c05ad731a23e32c90cdeb8efb1bc7ca99b993aba89d06fbf03214b742a"}
Nov 02 03:23:29.933666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.933643    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn" event={"ID":"554b9b58-87b5-443e-908a-56c80a422a4e","Type":"ContainerDied","Data":"2fb811c78422160937029711e59f7c506bb744f604bade242c2d6498ee266def"}
Nov 02 03:23:29.933788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.933648    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"
Nov 02 03:23:29.942126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.942103    4407 scope.go:122] "RemoveContainer" containerID="5cce861eb307379ee7ad63bb176d097073a74d471a21e8592819315a0188921c"
Nov 02 03:23:29.953125 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.953108    4407 scope.go:122] "RemoveContainer" containerID="815ce265a1eb4aa57a1e02e635b0cbac12629a2bb2bba2af7b23eeb295a4de80"
Nov 02 03:23:29.980808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.980777    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:23:29.980929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:29.980846    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:23:30.044884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.044847    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:23:30.045034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.044901    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:23:30.098771 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.098733    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"]
Nov 02 03:23:30.104867 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.104840    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-swgkh"]
Nov 02 03:23:30.217493 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.217463    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"]
Nov 02 03:23:30.222077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.222048    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-8tczn"]
Nov 02 03:23:30.339943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.332984    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"]
Nov 02 03:23:30.343524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.342154    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-j6cr2"]
Nov 02 03:23:30.356057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.355839    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" path="/var/lib/kubelet/pods/3d358326-31d3-477c-85cc-5e5069e03e16/volumes"
Nov 02 03:23:30.357163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.356724    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" path="/var/lib/kubelet/pods/554b9b58-87b5-443e-908a-56c80a422a4e/volumes"
Nov 02 03:23:30.358292 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.357373    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" path="/var/lib/kubelet/pods/587a7c84-880a-4726-bc56-bf1e47c2a98c/volumes"
Nov 02 03:23:30.358491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.358470    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" path="/var/lib/kubelet/pods/75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf/volumes"
Nov 02 03:23:30.450373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.450348    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:23:30.475936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.475914    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:23:30.562305 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562273    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-credential-socket\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.562492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562352    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-certs\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.562492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562486    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-istio-token\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-istio-token\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.562613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562541    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-credential-socket" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.562613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562587    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6393d363-3485-4df1-ad45-781181c46561-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6393d363-3485-4df1-ad45-781181c46561-istiod-ca-cert\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.562759 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562701    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-envoy\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.562759 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562717    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-certs" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.562844 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562759    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6393d363-3485-4df1-ad45-781181c46561-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6393d363-3485-4df1-ad45-781181c46561-istio-podinfo\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.562884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562841    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-kube-api-access-d4zvh\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-kube-api-access-d4zvh\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.562935 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562882    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-socket\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.562935 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562917    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-data\") pod \"6393d363-3485-4df1-ad45-781181c46561\" (UID: \"6393d363-3485-4df1-ad45-781181c46561\") "
Nov 02 03:23:30.563040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.562964    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6393d363-3485-4df1-ad45-781181c46561-istiod-ca-cert" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:23:30.563196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.563167    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-socket" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.563297 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.563277    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-data" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.563827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.563806    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.564266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.563835    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.564266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.563850    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6393d363-3485-4df1-ad45-781181c46561-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.564266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.563864    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.564266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.563879    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.564933 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.564911    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-istio-token" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:30.565026 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.564985    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-envoy" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.565514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.565489    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6393d363-3485-4df1-ad45-781181c46561-istio-podinfo" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:23:30.565581 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.565511    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-kube-api-access-d4zvh" pod "6393d363-3485-4df1-ad45-781181c46561" (UID: "6393d363-3485-4df1-ad45-781181c46561"). InnerVolumeSpecName "kube-api-access-d4zvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:30.665219 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665188    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7b1b168-b57f-437e-b114-512c12f4c753-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b1b168-b57f-437e-b114-512c12f4c753-istio-podinfo\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.665402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665269    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-envoy\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.665402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665329    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-certs\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.665402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665383    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-credential-socket\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.665579 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665451    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-data\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.665656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665583    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-kube-api-access-hqmpg\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-kube-api-access-hqmpg\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.665729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665669    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-istio-token\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-istio-token\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.665729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665701    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-certs" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.665842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665733    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-credential-socket" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.665842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665778    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-socket\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.665842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665828    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-data" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.665842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.665837    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7b1b168-b57f-437e-b114-512c12f4c753-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b1b168-b57f-437e-b114-512c12f4c753-istiod-ca-cert\") pod \"a7b1b168-b57f-437e-b114-512c12f4c753\" (UID: \"a7b1b168-b57f-437e-b114-512c12f4c753\") "
Nov 02 03:23:30.666214 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666091    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-socket" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.666711 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666612    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b1b168-b57f-437e-b114-512c12f4c753-istiod-ca-cert" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:23:30.666847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666721    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.666847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666744    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.666847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666759    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6393d363-3485-4df1-ad45-781181c46561-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.666847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666772    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6393d363-3485-4df1-ad45-781181c46561-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.666847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666787    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4zvh\" (UniqueName: \"kubernetes.io/projected/6393d363-3485-4df1-ad45-781181c46561-kube-api-access-d4zvh\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.666847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666800    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.666847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666814    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.666847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.666826    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.667739 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.667706    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7b1b168-b57f-437e-b114-512c12f4c753-istio-podinfo" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:23:30.667890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.667871    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-envoy" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:23:30.668042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.668028    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-istio-token" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:30.668500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.668475    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-kube-api-access-hqmpg" pod "a7b1b168-b57f-437e-b114-512c12f4c753" (UID: "a7b1b168-b57f-437e-b114-512c12f4c753"). InnerVolumeSpecName "kube-api-access-hqmpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:23:30.768465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.768432    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqmpg\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-kube-api-access-hqmpg\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.768465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.768466    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b1b168-b57f-437e-b114-512c12f4c753-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.768465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.768480    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b1b168-b57f-437e-b114-512c12f4c753-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.768791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.768492    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b1b168-b57f-437e-b114-512c12f4c753-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.768791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.768505    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b1b168-b57f-437e-b114-512c12f4c753-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:23:30.941105 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.941072    4407 generic.go:356] "Generic (PLEG): container finished" podID="6393d363-3485-4df1-ad45-781181c46561" containerID="7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f" exitCode=0
Nov 02 03:23:30.941594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.941128    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"
Nov 02 03:23:30.941594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.941153    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" event={"ID":"6393d363-3485-4df1-ad45-781181c46561","Type":"ContainerDied","Data":"7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f"}
Nov 02 03:23:30.941594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.941227    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn" event={"ID":"6393d363-3485-4df1-ad45-781181c46561","Type":"ContainerDied","Data":"e2088963dbee2d791f0dfd434a309ccc9e2bcedd07321e445c5481cafc70f508"}
Nov 02 03:23:30.941594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.941260    4407 scope.go:122] "RemoveContainer" containerID="7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f"
Nov 02 03:23:30.945881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.945859    4407 generic.go:356] "Generic (PLEG): container finished" podID="a7b1b168-b57f-437e-b114-512c12f4c753" containerID="21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1" exitCode=0
Nov 02 03:23:30.945981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.945901    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"
Nov 02 03:23:30.945981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.945944    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" event={"ID":"a7b1b168-b57f-437e-b114-512c12f4c753","Type":"ContainerDied","Data":"21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1"}
Nov 02 03:23:30.946065 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.945982    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5" event={"ID":"a7b1b168-b57f-437e-b114-512c12f4c753","Type":"ContainerDied","Data":"aa78a65e5a8701905edb3ae21fec0a1740b7cf3473224efa7413669a433fdd94"}
Nov 02 03:23:30.953220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.953201    4407 scope.go:122] "RemoveContainer" containerID="7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f"
Nov 02 03:23:30.953493 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:23:30.953472    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f\": container with ID starting with 7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f not found: ID does not exist" containerID="7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f"
Nov 02 03:23:30.953569 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.953509    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f"} err="failed to get container status \"7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f\": rpc error: code = NotFound desc = could not find container \"7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f\": container with ID starting with 7f3b7a0e78c69f6a046c74acddeb4ae4699a1a97341e43e4d7d3c895d7f3da9f not found: ID does not exist"
Nov 02 03:23:30.953569 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.953526    4407 scope.go:122] "RemoveContainer" containerID="21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1"
Nov 02 03:23:30.965289 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.965269    4407 scope.go:122] "RemoveContainer" containerID="21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1"
Nov 02 03:23:30.965565 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:23:30.965537    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1\": container with ID starting with 21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1 not found: ID does not exist" containerID="21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1"
Nov 02 03:23:30.965676 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:30.965575    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1"} err="failed to get container status \"21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1\": rpc error: code = NotFound desc = could not find container \"21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1\": container with ID starting with 21b37d3dc0148c89e075635788e51709c0ca24c61fa0405b97160554a1dff9d1 not found: ID does not exist"
Nov 02 03:23:31.109644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:31.109546    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"]
Nov 02 03:23:31.116243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:31.116215    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-hrftn"]
Nov 02 03:23:31.227287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:31.227240    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"]
Nov 02 03:23:31.230316 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:31.230288    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-zrgw5"]
Nov 02 03:23:32.353381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:32.353349    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6393d363-3485-4df1-ad45-781181c46561" path="/var/lib/kubelet/pods/6393d363-3485-4df1-ad45-781181c46561/volumes"
Nov 02 03:23:32.353898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:32.353882    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" path="/var/lib/kubelet/pods/a7b1b168-b57f-437e-b114-512c12f4c753/volumes"
Nov 02 03:23:32.909168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:32.909141    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:23:34.478914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:34.478886    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:35.189830 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:23:35.189789    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:23:37.162419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:37.162388    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:39.803445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:39.803415    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:40.392723 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:40.392691    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:45.146238 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:45.146199    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:45.818171 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:45.818138    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:46.387908 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.387874    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:46.860245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860212    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"]
Nov 02 03:23:46.860814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860801    4407 cpu_manager.go:414] "containerMap: removing container" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy"
Nov 02 03:23:46.860856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860816    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy"
Nov 02 03:23:46.860856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860842    4407 cpu_manager.go:414] "containerMap: removing container" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy"
Nov 02 03:23:46.860856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860847    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy"
Nov 02 03:23:46.860856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860856    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy"
Nov 02 03:23:46.860989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860861    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy"
Nov 02 03:23:46.860989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860869    4407 cpu_manager.go:414] "containerMap: removing container" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy"
Nov 02 03:23:46.860989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860873    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy"
Nov 02 03:23:46.860989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860882    4407 cpu_manager.go:414] "containerMap: removing container" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy"
Nov 02 03:23:46.860989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860887    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy"
Nov 02 03:23:46.860989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860907    4407 cpu_manager.go:414] "containerMap: removing container" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy"
Nov 02 03:23:46.860989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860911    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy"
Nov 02 03:23:46.860989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.860991    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="587a7c84-880a-4726-bc56-bf1e47c2a98c" containerName="istio-proxy"
Nov 02 03:23:46.861209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.861000    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b1b168-b57f-437e-b114-512c12f4c753" containerName="istio-proxy"
Nov 02 03:23:46.861209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.861008    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d358326-31d3-477c-85cc-5e5069e03e16" containerName="istio-proxy"
Nov 02 03:23:46.861209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.861013    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="554b9b58-87b5-443e-908a-56c80a422a4e" containerName="istio-proxy"
Nov 02 03:23:46.861209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.861021    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="6393d363-3485-4df1-ad45-781181c46561" containerName="istio-proxy"
Nov 02 03:23:46.861209 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.861028    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="75a05cd9-4e74-402e-b6ff-dd3bb7a8caaf" containerName="istio-proxy"
Nov 02 03:23:46.866099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.866079    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:46.940169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.940130    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"]
Nov 02 03:23:46.976902 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:46.976868    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-c170e9-openshift-default-dockercfg-v9gmp\""
Nov 02 03:23:47.018179 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.018142    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:47.025325 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.025286    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4hk6\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-kube-api-access-b4hk6\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.028649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.025677    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-podinfo\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.028649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.025728    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-socket\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.028649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.025771    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-token\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.028649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.025924    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-certs\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.028649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.026097    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-credential-socket\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.028649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.026163    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-data\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.028649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.026233    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d5530f-010d-4fc8-b40f-79c2eb27d763-istiod-ca-cert\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.028649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.026346    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-envoy\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128113 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128009    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-certs\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128113    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-credential-socket\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128167    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-data\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128246    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d5530f-010d-4fc8-b40f-79c2eb27d763-istiod-ca-cert\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128385    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-envoy\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128432    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4hk6\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-kube-api-access-b4hk6\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128472    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-podinfo\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128471    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-certs\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128499    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-socket\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.128776 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.128535    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-token\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.129451 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.129427    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-credential-socket\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.129728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.129707    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-data\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.131299 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.130943    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-socket\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.131299 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.131242    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d5530f-010d-4fc8-b40f-79c2eb27d763-istiod-ca-cert\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.132325 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.132304    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-envoy\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.134640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.134579    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-podinfo\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.186265 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.186235    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-token\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.186411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.186309    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4hk6\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-kube-api-access-b4hk6\") pod \"e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") " pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.344962 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.344924    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:47.481840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.481749    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:47.912061 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.912032    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"]
Nov 02 03:23:47.912807 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:23:47.912780    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14d5530f_010d_4fc8_b40f_79c2eb27d763.slice/crio-3450eb1222671d7a741f5e1cf5c6b0b9b06c31f8801aa60c1e9f963bcf8dfdc0 WatchSource:0}: Error finding container 3450eb1222671d7a741f5e1cf5c6b0b9b06c31f8801aa60c1e9f963bcf8dfdc0: Status 404 returned error can't find the container with id 3450eb1222671d7a741f5e1cf5c6b0b9b06c31f8801aa60c1e9f963bcf8dfdc0
Nov 02 03:23:47.915578 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.915545    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:23:47.915686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.915617    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:23:47.915686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:47.915669    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:23:48.069512 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:48.069471    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" event={"ID":"14d5530f-010d-4fc8-b40f-79c2eb27d763","Type":"ContainerStarted","Data":"392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900"}
Nov 02 03:23:48.069719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:48.069539    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" event={"ID":"14d5530f-010d-4fc8-b40f-79c2eb27d763","Type":"ContainerStarted","Data":"3450eb1222671d7a741f5e1cf5c6b0b9b06c31f8801aa60c1e9f963bcf8dfdc0"}
Nov 02 03:23:48.186501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:48.186392    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" podStartSLOduration=2.186376007 podStartE2EDuration="2.186376007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:23:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:23:48.185874803 +0000 UTC m=+2171.981223369" watchObservedRunningTime="2026-11-02 03:23:48.186376007 +0000 UTC m=+2171.981724574"
Nov 02 03:23:48.482433 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:48.482356    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:48.488023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:48.487999    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:49.075980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:49.075954    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:49.076929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:49.076910    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:23:50.549520 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:50.549488    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:50.956005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:50.955920    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:51.131596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:51.131547    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:52.860789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:52.860757    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:53.142061 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:53.141973    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:53.298819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:53.298782    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:23:56.007423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:23:56.007389    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:00.065740 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:00.065701    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:00.232583 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:00.232534    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:01.561041 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:01.561014    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:04.194485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:04.194451    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:24:05.214872 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:24:05.214828    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:24:06.407178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:06.407142    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:06.557943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:06.557913    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:11.239227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:11.239192    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:11.476276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:11.476198    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:11.637410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:11.637377    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:14.261813 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:14.261778    4407 generic.go:356] "Generic (PLEG): container finished" podID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerID="15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a" exitCode=0
Nov 02 03:24:14.262297 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:14.261852    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" event={"ID":"e6e61c64-563d-4c06-b771-89b52ff8191b","Type":"ContainerDied","Data":"15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a"}
Nov 02 03:24:14.262425 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:14.262407    4407 scope.go:122] "RemoveContainer" containerID="15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a"
Nov 02 03:24:14.264098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:14.264070    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:24:14.264163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:14.264147    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:24:14.264198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:14.264181    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:24:15.270423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:15.270390    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" event={"ID":"e6e61c64-563d-4c06-b771-89b52ff8191b","Type":"ContainerStarted","Data":"d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e"}
Nov 02 03:24:15.653544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:15.653445    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:24:15.654600 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:15.654572    4407 patch_prober.go:29] interesting pod/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 03:24:15.654745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:15.654651    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 03:24:16.300711 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:16.298081    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:16.653327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:16.653251    4407 patch_prober.go:29] interesting pod/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 03:24:16.653327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:16.653300    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 03:24:17.652873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:17.652837    4407 patch_prober.go:29] interesting pod/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 03:24:17.653350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:17.652900    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 03:24:18.652836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:18.652796    4407 patch_prober.go:29] interesting pod/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 03:24:18.653041 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:18.652889    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 03:24:19.652660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:19.652603    4407 patch_prober.go:29] interesting pod/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 03:24:19.652837 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:19.652682    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 03:24:20.331309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:20.331281    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:24:20.657340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:20.657254    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:24:21.180049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:21.180014    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:21.327775 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:21.327743    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:24:25.965883 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:25.965843    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:30.889088 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:30.889057    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:34.926296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:34.926256    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:35.214167 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:24:35.214082    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:24:36.073179 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:36.073146    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:36.371806 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:36.371728    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:36.529739 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:36.529698    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:40.727158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:40.727129    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:41.300565 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:41.300535    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:46.470226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:46.470192    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:48.860345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:48.860316    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:49.020499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:49.020428    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:51.595049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:51.595010    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:51.770411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:51.770380    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:51.946373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:51.946289    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:24:56.786707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:24:56.786613    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:00.540733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.540705    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:25:00.763684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.763651    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"]
Nov 02 03:25:00.769058 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.769037    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.831591 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.831501    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.831591 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.831559    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8jsz\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-kube-api-access-q8jsz\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.831802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.831711    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.831802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.831773    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.831976 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.831961    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.832040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.832000    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.832040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.832027    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.832145 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.832118    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.832195 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.832146    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933129    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8jsz\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-kube-api-access-q8jsz\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933189    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933215    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933344    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933372    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933391    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933420    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933443    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933534    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933694    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933930 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933817    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933920    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.933971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933954    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.934064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.933992    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.935909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.935881    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.936092 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.936076    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.968592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.968511    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"]
Nov 02 03:25:00.990972 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.990945    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:00.991452 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:00.991427    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8jsz\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-kube-api-access-q8jsz\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:01.083146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:01.083062    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:01.513683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:01.513610    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"]
Nov 02 03:25:01.551712 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:25:01.551686    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb62fe9_84bd_4362_9bca_d179e60db6d0.slice/crio-bcfe2823d7d3df09a3cb9f9b187f33b0926684395c3127811bca843de1cf21ae WatchSource:0}: Error finding container bcfe2823d7d3df09a3cb9f9b187f33b0926684395c3127811bca843de1cf21ae: Status 404 returned error can't find the container with id bcfe2823d7d3df09a3cb9f9b187f33b0926684395c3127811bca843de1cf21ae
Nov 02 03:25:01.554170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:01.554134    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:01.554281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:01.554217    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:01.554281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:01.554264    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:01.562342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:01.562320    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:01.617304 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:01.617273    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" event={"ID":"3eb62fe9-84bd-4362-9bca-d179e60db6d0","Type":"ContainerStarted","Data":"bcfe2823d7d3df09a3cb9f9b187f33b0926684395c3127811bca843de1cf21ae"}
Nov 02 03:25:02.361031 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:02.360997    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:02.532032 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:02.531998    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:02.625028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:02.624943    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" event={"ID":"3eb62fe9-84bd-4362-9bca-d179e60db6d0","Type":"ContainerStarted","Data":"171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a"}
Nov 02 03:25:02.832334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:02.832274    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" podStartSLOduration=2.832261655 podStartE2EDuration="2.832261655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:25:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:25:02.803101284 +0000 UTC m=+2246.598449850" watchObservedRunningTime="2026-11-02 03:25:02.832261655 +0000 UTC m=+2246.627610220"
Nov 02 03:25:03.101649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:03.098959    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:03.105802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:03.105736    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 03:25:03.105802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:03.105798    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 03:25:04.090651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:04.086378    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 03:25:04.090651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:04.086446    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 03:25:05.100649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:05.093102    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 03:25:05.100649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:05.093161    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 03:25:05.216545 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:05.216510    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:25:06.085978 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:06.085944    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 03:25:06.086599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:06.086231    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 03:25:06.403667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:06.403555    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:07.096651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:07.093086    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 03:25:07.096651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:07.093146    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 03:25:08.088205 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:08.088172    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:08.088615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:08.088550    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:08.089182 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:08.089167    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:25:11.164567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:11.164535    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:15.878161 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.878130    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"]
Nov 02 03:25:15.882635 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.882602    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.968749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.968710    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94c76785-3504-4459-9980-245d9faf185d-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.968945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.968875    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.968945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.968909    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.969135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.969110    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.969293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.969185    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tjsg\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-kube-api-access-4tjsg\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.969293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.969277    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.969402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.969333    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.969455 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.969444    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.969519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.969474    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94c76785-3504-4459-9980-245d9faf185d-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:15.986794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:15.986754    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:16.071262 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071227    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071269    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071342    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071374    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tjsg\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-kube-api-access-4tjsg\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071444    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071496    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071613    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071665    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94c76785-3504-4459-9980-245d9faf185d-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071719    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-certs\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071742    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94c76785-3504-4459-9980-245d9faf185d-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071753    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-data\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.071870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.071802    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.072162 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.072141    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-credential-socket\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.072437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.072402    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94c76785-3504-4459-9980-245d9faf185d-istiod-ca-cert\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.074349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.074326    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-envoy\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.074482 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.074465    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94c76785-3504-4459-9980-245d9faf185d-istio-podinfo\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.084970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.084944    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"]
Nov 02 03:25:16.135431 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.135336    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-istio-token\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.137812 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.137788    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tjsg\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-kube-api-access-4tjsg\") pod \"e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") " pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.198337 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.198305    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:16.626704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.626671    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"]
Nov 02 03:25:16.664872 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:25:16.664844    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c76785_3504_4459_9980_245d9faf185d.slice/crio-f2dd590a39d4148867d517d90de2c189818eb49755636dc65cc1f4aa79f3d832 WatchSource:0}: Error finding container f2dd590a39d4148867d517d90de2c189818eb49755636dc65cc1f4aa79f3d832: Status 404 returned error can't find the container with id f2dd590a39d4148867d517d90de2c189818eb49755636dc65cc1f4aa79f3d832
Nov 02 03:25:16.667591 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.667553    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:16.667681 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.667645    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:16.667728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.667679    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:16.718214 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:16.718183    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" event={"ID":"94c76785-3504-4459-9980-245d9faf185d","Type":"ContainerStarted","Data":"f2dd590a39d4148867d517d90de2c189818eb49755636dc65cc1f4aa79f3d832"}
Nov 02 03:25:17.726924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:17.726879    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" event={"ID":"94c76785-3504-4459-9980-245d9faf185d","Type":"ContainerStarted","Data":"574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1"}
Nov 02 03:25:17.905649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:17.905009    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" podStartSLOduration=2.9049883149999998 podStartE2EDuration="2.904988315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:25:17.90122573 +0000 UTC m=+2261.696574296" watchObservedRunningTime="2026-11-02 03:25:17.904988315 +0000 UTC m=+2261.700336876"
Nov 02 03:25:18.204819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:18.204783    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:18.205801 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:18.205774    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 03:25:18.205955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:18.205838    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 03:25:19.203648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:19.202033    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 03:25:19.203648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:19.202099    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 03:25:20.200647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:20.199986    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 03:25:20.200647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:20.200047    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 03:25:21.204648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:21.204114    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 03:25:21.204648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:21.204183    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 03:25:21.365995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:21.365965    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:22.201384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:22.201351    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 03:25:22.201725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:22.201693    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 03:25:23.199648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:23.199041    4407 patch_prober.go:29] interesting pod/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 03:25:23.199648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:23.199104    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 03:25:24.210534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:24.210501    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:24.211011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:24.210993    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:24.211666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:24.211645    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:25:25.847042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:25.847009    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"]
Nov 02 03:25:25.847464 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:25.847221    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7" podUID="60e5f97c-02aa-4d35-abed-284be1654094" containerName="maas-api" containerID="cri-o://2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9" gracePeriod=30
Nov 02 03:25:26.008430 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.008398    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:26.138019 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.137990    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:25:26.241052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.241025    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60e5f97c-02aa-4d35-abed-284be1654094-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e5f97c-02aa-4d35-abed-284be1654094-openshift-service-ca\") pod \"60e5f97c-02aa-4d35-abed-284be1654094\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") "
Nov 02 03:25:26.241217 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.241065    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-metrics-certs\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-metrics-certs\") pod \"60e5f97c-02aa-4d35-abed-284be1654094\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") "
Nov 02 03:25:26.241217 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.241199    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60e5f97c-02aa-4d35-abed-284be1654094-kube-api-access-s2zl2\" (UniqueName: \"kubernetes.io/projected/60e5f97c-02aa-4d35-abed-284be1654094-kube-api-access-s2zl2\") pod \"60e5f97c-02aa-4d35-abed-284be1654094\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") "
Nov 02 03:25:26.241345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.241218    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls\") pod \"60e5f97c-02aa-4d35-abed-284be1654094\" (UID: \"60e5f97c-02aa-4d35-abed-284be1654094\") "
Nov 02 03:25:26.241470 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.241442    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e5f97c-02aa-4d35-abed-284be1654094-openshift-service-ca" pod "60e5f97c-02aa-4d35-abed-284be1654094" (UID: "60e5f97c-02aa-4d35-abed-284be1654094"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:25:26.242808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.242772    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e5f97c-02aa-4d35-abed-284be1654094-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:26.243856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.243810    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-metrics-certs" pod "60e5f97c-02aa-4d35-abed-284be1654094" (UID: "60e5f97c-02aa-4d35-abed-284be1654094"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:25:26.243856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.243844    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e5f97c-02aa-4d35-abed-284be1654094-kube-api-access-s2zl2" pod "60e5f97c-02aa-4d35-abed-284be1654094" (UID: "60e5f97c-02aa-4d35-abed-284be1654094"). InnerVolumeSpecName "kube-api-access-s2zl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:26.244021 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.243927    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls" pod "60e5f97c-02aa-4d35-abed-284be1654094" (UID: "60e5f97c-02aa-4d35-abed-284be1654094"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:25:26.343976 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.343927    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2zl2\" (UniqueName: \"kubernetes.io/projected/60e5f97c-02aa-4d35-abed-284be1654094-kube-api-access-s2zl2\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:26.343976 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.343963    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:26.343976 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.343976    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60e5f97c-02aa-4d35-abed-284be1654094-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:26.796357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.796325    4407 generic.go:356] "Generic (PLEG): container finished" podID="60e5f97c-02aa-4d35-abed-284be1654094" containerID="2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9" exitCode=0
Nov 02 03:25:26.796548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.796385    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"
Nov 02 03:25:26.796548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.796414    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7" event={"ID":"60e5f97c-02aa-4d35-abed-284be1654094","Type":"ContainerDied","Data":"2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9"}
Nov 02 03:25:26.796548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.796480    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7" event={"ID":"60e5f97c-02aa-4d35-abed-284be1654094","Type":"ContainerDied","Data":"7e2f69106edcb5ac9e56a29ef324cf033ab2e495c187832071e24def82f20832"}
Nov 02 03:25:26.796548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.796511    4407 scope.go:122] "RemoveContainer" containerID="2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9"
Nov 02 03:25:26.808795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.808776    4407 scope.go:122] "RemoveContainer" containerID="2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9"
Nov 02 03:25:26.809070 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:26.809049    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9\": container with ID starting with 2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9 not found: ID does not exist" containerID="2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9"
Nov 02 03:25:26.809123 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.809083    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9"} err="failed to get container status \"2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9\": rpc error: code = NotFound desc = could not find container \"2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9\": container with ID starting with 2f25290026bb28f42ab5ecff2083b66fea2e5600345b327640dece1acd75a4e9 not found: ID does not exist"
Nov 02 03:25:26.979689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:26.978758    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"]
Nov 02 03:25:27.020919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:27.020880    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ed15c1-69868c8965-s6mn7"]
Nov 02 03:25:27.881649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:27.877191    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:28.093738 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:28.093700    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:28.353905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:28.353872    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60e5f97c-02aa-4d35-abed-284be1654094" path="/var/lib/kubelet/pods/60e5f97c-02aa-4d35-abed-284be1654094/volumes"
Nov 02 03:25:28.375582 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:28.375555    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:29.096452 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:29.096421    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:30.824575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:30.824543    4407 generic.go:356] "Generic (PLEG): container finished" podID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerID="392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900" exitCode=0
Nov 02 03:25:30.825136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:30.824614    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" event={"ID":"14d5530f-010d-4fc8-b40f-79c2eb27d763","Type":"ContainerDied","Data":"392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900"}
Nov 02 03:25:30.825207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:30.825174    4407 scope.go:122] "RemoveContainer" containerID="392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900"
Nov 02 03:25:30.826840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:30.826813    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:30.826902 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:30.826880    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:30.826941 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:30.826913    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:25:31.833074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:31.833037    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" event={"ID":"14d5530f-010d-4fc8-b40f-79c2eb27d763","Type":"ContainerStarted","Data":"fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f"}
Nov 02 03:25:32.452856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:32.452810    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:32.505884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:32.505842    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:25:32.507341 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:32.507315    4407 patch_prober.go:29] interesting pod/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 03:25:32.507471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:32.507366    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 03:25:33.494530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:33.494030    4407 patch_prober.go:29] interesting pod/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 03:25:33.494530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:33.494130    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 03:25:34.076521 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:34.076491    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:25:34.304206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:34.304169    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:34.502803 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:34.502768    4407 patch_prober.go:29] interesting pod/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 03:25:34.503428 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:34.503398    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 03:25:34.507666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:34.507563    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:34.713925 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:34.713892    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:34.964923 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:34.964842    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:35.143380 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:35.143156    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:35.220298 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:35.220203    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"]
Nov 02 03:25:35.220903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:35.220867    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy" containerID="cri-o://6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483" gracePeriod=30
Nov 02 03:25:35.232445 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:35.232418    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:25:35.327407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:35.327327    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:35.496712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:35.484993    4407 patch_prober.go:29] interesting pod/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 03:25:35.496712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:35.485062    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 03:25:35.574906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:35.574873    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"]
Nov 02 03:25:35.575345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:35.575064    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy" containerID="cri-o://0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c" gracePeriod=30
Nov 02 03:25:36.501649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:36.495069    4407 patch_prober.go:29] interesting pod/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 03:25:36.501649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:36.495131    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 03:25:37.486788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:37.486758    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:25:37.487744 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:37.487725    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:25:38.548171 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:38.548139    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"]
Nov 02 03:25:38.548564 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:38.548326    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy" containerID="cri-o://56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784" gracePeriod=30
Nov 02 03:25:38.601241 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:38.601209    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"]
Nov 02 03:25:38.601710 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:38.601678    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy" containerID="cri-o://a18c4dd741414da31a1193ef14892af8baf1515aca26a385fbd0537ac5bf0f65" gracePeriod=30
Nov 02 03:25:38.785218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:38.785183    4407 patch_prober.go:29] interesting pod/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:25:38.785392 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:38.785238    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:25:39.676607 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:39.676579    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:40.309020 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.308985    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:25:40.309195 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.309046    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:25:40.499935 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.499906    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:25:40.632844 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.632812    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-token\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-token\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.632959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.632882    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-credential-socket\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.633010 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.632985    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-kube-api-access-q9xck\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-kube-api-access-q9xck\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.633050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633032    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-data\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.633084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633074    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istiod-ca-cert\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.633127 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633113    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-podinfo\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.633229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633195    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-envoy\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.633315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633238    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-socket\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.633315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633276    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-certs\") pod \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\" (UID: \"c42b386b-9a17-4c21-b5f4-7cc0934d358a\") "
Nov 02 03:25:40.633428 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633194    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-credential-socket" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.633428 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633329    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-data" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.633596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633538    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-socket" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.633596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633453    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istiod-ca-cert" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:25:40.633792 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633774    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-certs" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.633983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633962    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.634054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.633991    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.634054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.634006    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.634054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.634021    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.634054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.634035    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.635768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.635748    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-token" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:40.635839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.635763    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-kube-api-access-q9xck" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "kube-api-access-q9xck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:40.635975 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.635960    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-podinfo" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:25:40.636018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.635977    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-envoy" pod "c42b386b-9a17-4c21-b5f4-7cc0934d358a" (UID: "c42b386b-9a17-4c21-b5f4-7cc0934d358a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.735596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.735557    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.735596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.735587    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.735596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.735597    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.735596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.735608    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9xck\" (UniqueName: \"kubernetes.io/projected/c42b386b-9a17-4c21-b5f4-7cc0934d358a-kube-api-access-q9xck\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.821018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.820993    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:25:40.902342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.902300    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:40.904158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.904135    4407 generic.go:356] "Generic (PLEG): container finished" podID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerID="6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483" exitCode=0
Nov 02 03:25:40.904294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.904198    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"
Nov 02 03:25:40.904294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.904223    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" event={"ID":"c42b386b-9a17-4c21-b5f4-7cc0934d358a","Type":"ContainerDied","Data":"6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483"}
Nov 02 03:25:40.904294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.904278    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68" event={"ID":"c42b386b-9a17-4c21-b5f4-7cc0934d358a","Type":"ContainerDied","Data":"feb3c970cea72a181be83aedfa8865847c42cf7a6c7dc7ee1add0c6848d8cb97"}
Nov 02 03:25:40.904461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.904308    4407 scope.go:122] "RemoveContainer" containerID="6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483"
Nov 02 03:25:40.905732 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.905707    4407 generic.go:356] "Generic (PLEG): container finished" podID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerID="0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c" exitCode=0
Nov 02 03:25:40.905824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.905755    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"
Nov 02 03:25:40.905824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.905761    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" event={"ID":"5e8e57a2-8970-4f43-ad20-ed3a3642268c","Type":"ContainerDied","Data":"0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c"}
Nov 02 03:25:40.905824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.905800    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz" event={"ID":"5e8e57a2-8970-4f43-ad20-ed3a3642268c","Type":"ContainerDied","Data":"1fdb291d40f395b4a6a23196f2a540932c8d5bad271958f4dc660db3116b640a"}
Nov 02 03:25:40.916984 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.916961    4407 scope.go:122] "RemoveContainer" containerID="3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c"
Nov 02 03:25:40.926463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.926448    4407 scope.go:122] "RemoveContainer" containerID="6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483"
Nov 02 03:25:40.926764 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:40.926740    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483\": container with ID starting with 6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483 not found: ID does not exist" containerID="6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483"
Nov 02 03:25:40.926859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.926772    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483"} err="failed to get container status \"6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483\": rpc error: code = NotFound desc = could not find container \"6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483\": container with ID starting with 6b274a67818c11691c5da4105ecf6b6d98b75385e6d347a2fa6c63b09c637483 not found: ID does not exist"
Nov 02 03:25:40.926859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.926790    4407 scope.go:122] "RemoveContainer" containerID="3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c"
Nov 02 03:25:40.927030 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:40.927013    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c\": container with ID starting with 3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c not found: ID does not exist" containerID="3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c"
Nov 02 03:25:40.927068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.927036    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c"} err="failed to get container status \"3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c\": rpc error: code = NotFound desc = could not find container \"3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c\": container with ID starting with 3d1e959e09646c7aaedf142f97a4cd1bb9564450d3d879e19ff373586754d14c not found: ID does not exist"
Nov 02 03:25:40.927068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.927050    4407 scope.go:122] "RemoveContainer" containerID="0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c"
Nov 02 03:25:40.936396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.936372    4407 scope.go:122] "RemoveContainer" containerID="0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c"
Nov 02 03:25:40.936698 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:40.936674    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c\": container with ID starting with 0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c not found: ID does not exist" containerID="0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c"
Nov 02 03:25:40.936767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.936707    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c"} err="failed to get container status \"0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c\": rpc error: code = NotFound desc = could not find container \"0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c\": container with ID starting with 0014cfb5e85ca0d886f9dd14cba68302e2ba5540b6e21aef3a397c54ad17081c not found: ID does not exist"
Nov 02 03:25:40.937183 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937168    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-kube-api-access-b9l7v\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-kube-api-access-b9l7v\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937201    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-envoy\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937232    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-credential-socket\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937377 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937363    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-socket\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937439 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937402    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-podinfo\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937445    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istiod-ca-cert\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937449    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-credential-socket" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.937606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937506    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-certs\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937535    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-data\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937578    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-token\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-token\") pod \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\" (UID: \"5e8e57a2-8970-4f43-ad20-ed3a3642268c\") "
Nov 02 03:25:40.937770 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937606    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-socket" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.937832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937812    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-data" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.937926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.937905    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istiod-ca-cert" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:25:40.938084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.938058    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-certs" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.938551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.938518    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.938551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.938545    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.938740 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.938561    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.938740 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.938575    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.938740 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.938589    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:40.940229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.940204    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-envoy" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:40.940420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.940216    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-podinfo" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:25:40.940420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.940238    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-kube-api-access-b9l7v" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "kube-api-access-b9l7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:40.940420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:40.940266    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-token" pod "5e8e57a2-8970-4f43-ad20-ed3a3642268c" (UID: "5e8e57a2-8970-4f43-ad20-ed3a3642268c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:41.040326 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.040236    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9l7v\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-kube-api-access-b9l7v\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:41.040326 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.040268    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:41.040326 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.040281    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:41.040326 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.040289    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e8e57a2-8970-4f43-ad20-ed3a3642268c-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:41.090845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.090815    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:41.147918 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.147881    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"]
Nov 02 03:25:41.155127 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.155088    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ed15c1-openshift-default-656fd8c59-6rn68"]
Nov 02 03:25:41.177157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.177128    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"]
Nov 02 03:25:41.177442 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.177403    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy" containerID="cri-o://029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608" gracePeriod=30
Nov 02 03:25:41.256019 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.255989    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:41.437347 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.437264    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"]
Nov 02 03:25:41.444144 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.444110    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-n25hz"]
Nov 02 03:25:41.647334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:41.646133    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:25:42.352077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:42.352045    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" path="/var/lib/kubelet/pods/5e8e57a2-8970-4f43-ad20-ed3a3642268c/volumes"
Nov 02 03:25:42.352563 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:42.352549    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" path="/var/lib/kubelet/pods/c42b386b-9a17-4c21-b5f4-7cc0934d358a/volumes"
Nov 02 03:25:43.879322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.879281    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:25:43.893744 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.893714    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-token\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-token\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.893909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.893781    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-socket\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.893909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.893870    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-envoy\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.893909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.893897    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-kube-api-access-mcs7l\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-kube-api-access-mcs7l\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.894077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.893945    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-certs\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.894077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894002    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-podinfo\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.894077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894061    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istiod-ca-cert\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.894224 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894076    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-socket" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.894224 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894176    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-data\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.894336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894305    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-credential-socket\") pod \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\" (UID: \"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a\") "
Nov 02 03:25:43.894613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894292    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-certs" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.894773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894414    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istiod-ca-cert" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:25:43.894837 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894664    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-credential-socket" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.894837 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894734    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-data" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.895024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.894999    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.895135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.895028    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.895135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.895043    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.895135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.895056    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.895135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.895069    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.896427 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.896407    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-kube-api-access-mcs7l" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "kube-api-access-mcs7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:43.897636 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.897601    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-podinfo" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:25:43.897707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.897616    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-envoy" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.897707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.897653    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-token" pod "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" (UID: "d4cd0ea3-6d06-47f1-98a2-a4f94645a44a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:43.930384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.930347    4407 generic.go:356] "Generic (PLEG): container finished" podID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerID="56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784" exitCode=0
Nov 02 03:25:43.930544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.930464    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"
Nov 02 03:25:43.930544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.930471    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" event={"ID":"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a","Type":"ContainerDied","Data":"56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784"}
Nov 02 03:25:43.930650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.930540    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9" event={"ID":"d4cd0ea3-6d06-47f1-98a2-a4f94645a44a","Type":"ContainerDied","Data":"761c756e3231f8e0b0214033d878d59d59655de3f4c253336fc46dff8e4f2564"}
Nov 02 03:25:43.930650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.930571    4407 scope.go:122] "RemoveContainer" containerID="56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784"
Nov 02 03:25:43.932701 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.932668    4407 generic.go:356] "Generic (PLEG): container finished" podID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerID="a18c4dd741414da31a1193ef14892af8baf1515aca26a385fbd0537ac5bf0f65" exitCode=0
Nov 02 03:25:43.932788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.932719    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" event={"ID":"c17d8e32-b682-4c2e-8d56-b5aecc10042b","Type":"ContainerDied","Data":"a18c4dd741414da31a1193ef14892af8baf1515aca26a385fbd0537ac5bf0f65"}
Nov 02 03:25:43.941595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.941577    4407 scope.go:122] "RemoveContainer" containerID="56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784"
Nov 02 03:25:43.941901 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:43.941872    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784\": container with ID starting with 56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784 not found: ID does not exist" containerID="56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784"
Nov 02 03:25:43.941988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.941915    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784"} err="failed to get container status \"56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784\": rpc error: code = NotFound desc = could not find container \"56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784\": container with ID starting with 56145e99c3b87c6e3c517bc4d94ffb46579ea22093cca4cba4482034c592c784 not found: ID does not exist"
Nov 02 03:25:43.942308 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.942292    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:25:43.995525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.995498    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-certs\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.995715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.995652    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istiod-ca-cert\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.995776 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.995717    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-token\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-token\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.995776 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.995740    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-kube-api-access-lv729\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-kube-api-access-lv729\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.995892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.995800    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-certs" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.995892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.995858    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-podinfo\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.995997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.995948    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-socket\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.995997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.995988    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-envoy\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.996138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996021    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-credential-socket\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.996138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996093    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istiod-ca-cert" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:25:43.996242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996157    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-data\") pod \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\" (UID: \"c17d8e32-b682-4c2e-8d56-b5aecc10042b\") "
Nov 02 03:25:43.996242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996213    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-socket" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.996354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996332    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-credential-socket" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.996471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996443    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-data" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:43.996833 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996813    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.996926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996839    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.996926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996854    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.996926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996867    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.996926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996881    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.996926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996894    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.996926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996908    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcs7l\" (UniqueName: \"kubernetes.io/projected/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-kube-api-access-mcs7l\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.996926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996922    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.997185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.996936    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:43.998177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.998157    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-kube-api-access-lv729" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "kube-api-access-lv729". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:43.998274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.998256    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-podinfo" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:25:43.998637 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.998603    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-token" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:43.998637 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:43.998612    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-envoy" pod "c17d8e32-b682-4c2e-8d56-b5aecc10042b" (UID: "c17d8e32-b682-4c2e-8d56-b5aecc10042b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:44.098796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.098713    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:44.098796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.098744    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv729\" (UniqueName: \"kubernetes.io/projected/c17d8e32-b682-4c2e-8d56-b5aecc10042b-kube-api-access-lv729\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:44.098796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.098754    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:44.098796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.098763    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17d8e32-b682-4c2e-8d56-b5aecc10042b-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:44.163219 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.163188    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"]
Nov 02 03:25:44.168429 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.168405    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-ljzh9"]
Nov 02 03:25:44.348843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.348769    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:44.352378 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.352353    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" path="/var/lib/kubelet/pods/d4cd0ea3-6d06-47f1-98a2-a4f94645a44a/volumes"
Nov 02 03:25:44.522226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.522187    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:44.735169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.735066    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:44.949993 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.949963    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"
Nov 02 03:25:44.950474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.949984    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh" event={"ID":"c17d8e32-b682-4c2e-8d56-b5aecc10042b","Type":"ContainerDied","Data":"378fb0e8b3946e3a267f51cdbef476dd0bdfce5644d09dafcef786e3311f4abb"}
Nov 02 03:25:44.950474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:44.950121    4407 scope.go:122] "RemoveContainer" containerID="a18c4dd741414da31a1193ef14892af8baf1515aca26a385fbd0537ac5bf0f65"
Nov 02 03:25:45.116812 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:45.116750    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"]
Nov 02 03:25:45.167261 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:45.167227    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-4djkh"]
Nov 02 03:25:46.353427 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.353396    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" path="/var/lib/kubelet/pods/c17d8e32-b682-4c2e-8d56-b5aecc10042b/volumes"
Nov 02 03:25:46.462434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.462362    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:25:46.538273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538238    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istiod-ca-cert\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538298    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-data\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538400    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-kube-api-access-6lzd9\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-kube-api-access-6lzd9\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538463    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-podinfo\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538543    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-data" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:46.538701 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538567    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-credential-socket\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538701 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538591    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-token\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-token\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538701 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538673    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istiod-ca-cert" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:25:46.538881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538752    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-envoy\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538831    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-certs\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538862    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-socket\") pod \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\" (UID: \"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b\") "
Nov 02 03:25:46.538881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.538862    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-credential-socket" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:46.539090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.539066    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-certs" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:46.539277 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.539222    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-socket" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:46.539446 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.539429    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.539513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.539450    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.539513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.539459    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.539513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.539468    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.539513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.539476    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.541642 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.541595    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-envoy" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:25:46.541768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.541752    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-podinfo" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:25:46.541808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.541777    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-token" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:46.541850 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.541805    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-kube-api-access-6lzd9" pod "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" (UID: "b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b"). InnerVolumeSpecName "kube-api-access-6lzd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:25:46.640393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.640350    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lzd9\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-kube-api-access-6lzd9\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.640393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.640381    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.640393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.640394    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.640393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.640403    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:25:46.959399 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.959369    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:46.977612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.972044    4407 generic.go:356] "Generic (PLEG): container finished" podID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerID="029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608" exitCode=0
Nov 02 03:25:46.977612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.972110    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" event={"ID":"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b","Type":"ContainerDied","Data":"029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608"}
Nov 02 03:25:46.977612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.972147    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4" event={"ID":"b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b","Type":"ContainerDied","Data":"56bf2eb09d08b0981251181434cee456ac011e7d7123d2736ecd8f8b9b0b902b"}
Nov 02 03:25:46.977612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.972169    4407 scope.go:122] "RemoveContainer" containerID="029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608"
Nov 02 03:25:46.977612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:46.972258    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"
Nov 02 03:25:47.013166 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:47.011519    4407 scope.go:122] "RemoveContainer" containerID="88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01"
Nov 02 03:25:47.031682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:47.031659    4407 scope.go:122] "RemoveContainer" containerID="029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608"
Nov 02 03:25:47.032111 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:47.032035    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608\": container with ID starting with 029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608 not found: ID does not exist" containerID="029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608"
Nov 02 03:25:47.032111 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:47.032073    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608"} err="failed to get container status \"029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608\": rpc error: code = NotFound desc = could not find container \"029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608\": container with ID starting with 029775ef1540035bd4caf563650126cb31eea1edeaa5cf3c39c14d5ac3e4f608 not found: ID does not exist"
Nov 02 03:25:47.032111 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:47.032100    4407 scope.go:122] "RemoveContainer" containerID="88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01"
Nov 02 03:25:47.032434 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:25:47.032370    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01\": container with ID starting with 88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01 not found: ID does not exist" containerID="88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01"
Nov 02 03:25:47.032434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:47.032395    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01"} err="failed to get container status \"88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01\": rpc error: code = NotFound desc = could not find container \"88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01\": container with ID starting with 88a1324cf180ffcfd251fb280eb3059eb05d3ffb187b487d1dec7930ece45e01 not found: ID does not exist"
Nov 02 03:25:47.116384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:47.116325    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:47.144408 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:47.144369    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"]
Nov 02 03:25:47.149950 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:47.149915    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-0d26af-openshift-default-7d99cfb95-7vgc4"]
Nov 02 03:25:48.360318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:48.360104    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" path="/var/lib/kubelet/pods/b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b/volumes"
Nov 02 03:25:49.973564 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:49.973529    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:53.921222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:53.921189    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:54.070709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:54.070674    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:54.786528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:54.786492    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:25:59.615062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:25:59.615033    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:03.596777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:03.596727    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:04.774067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:04.774006    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:05.189775 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:26:05.189695    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:26:09.919483 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:09.919450    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:14.649842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:14.649812    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:15.268683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:15.268650    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:26:16.614589 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:16.614561    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:16.772099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:16.772064    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:18.893721 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:18.893673    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:20.462534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:20.462498    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:20.676074 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:20.676044    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:20.837959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:20.837929    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:21.360568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:21.360540    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:25.008540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.008507    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"]
Nov 02 03:26:25.008989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.008769    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l" podUID="245451d8-d8c5-4b7c-8445-3460460cf9fd" containerName="maas-api" containerID="cri-o://dd657d4a69ca5a7d61dd1c47996cbe7dda64910be045dbf684e4fc7c4c6f46dd" gracePeriod=30
Nov 02 03:26:25.136814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.136780    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:25.266907 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.266817    4407 generic.go:356] "Generic (PLEG): container finished" podID="245451d8-d8c5-4b7c-8445-3460460cf9fd" containerID="dd657d4a69ca5a7d61dd1c47996cbe7dda64910be045dbf684e4fc7c4c6f46dd" exitCode=0
Nov 02 03:26:25.267082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.266913    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l" event={"ID":"245451d8-d8c5-4b7c-8445-3460460cf9fd","Type":"ContainerDied","Data":"dd657d4a69ca5a7d61dd1c47996cbe7dda64910be045dbf684e4fc7c4c6f46dd"}
Nov 02 03:26:25.313140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.313115    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:26:25.442618 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.442584    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/245451d8-d8c5-4b7c-8445-3460460cf9fd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245451d8-d8c5-4b7c-8445-3460460cf9fd-openshift-service-ca\") pod \"245451d8-d8c5-4b7c-8445-3460460cf9fd\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") "
Nov 02 03:26:25.442834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.442665    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-metrics-certs\") pod \"245451d8-d8c5-4b7c-8445-3460460cf9fd\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") "
Nov 02 03:26:25.442834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.442810    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls\") pod \"245451d8-d8c5-4b7c-8445-3460460cf9fd\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") "
Nov 02 03:26:25.442962 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.442862    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/245451d8-d8c5-4b7c-8445-3460460cf9fd-kube-api-access-c46qw\" (UniqueName: \"kubernetes.io/projected/245451d8-d8c5-4b7c-8445-3460460cf9fd-kube-api-access-c46qw\") pod \"245451d8-d8c5-4b7c-8445-3460460cf9fd\" (UID: \"245451d8-d8c5-4b7c-8445-3460460cf9fd\") "
Nov 02 03:26:25.444769 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.444738    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245451d8-d8c5-4b7c-8445-3460460cf9fd-openshift-service-ca" pod "245451d8-d8c5-4b7c-8445-3460460cf9fd" (UID: "245451d8-d8c5-4b7c-8445-3460460cf9fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:26:25.446041 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.446011    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-metrics-certs" pod "245451d8-d8c5-4b7c-8445-3460460cf9fd" (UID: "245451d8-d8c5-4b7c-8445-3460460cf9fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:26:25.447920 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.447895    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls" pod "245451d8-d8c5-4b7c-8445-3460460cf9fd" (UID: "245451d8-d8c5-4b7c-8445-3460460cf9fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:26:25.448108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.448082    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245451d8-d8c5-4b7c-8445-3460460cf9fd-kube-api-access-c46qw" pod "245451d8-d8c5-4b7c-8445-3460460cf9fd" (UID: "245451d8-d8c5-4b7c-8445-3460460cf9fd"). InnerVolumeSpecName "kube-api-access-c46qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:26:25.545363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.545264    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245451d8-d8c5-4b7c-8445-3460460cf9fd-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:26:25.545363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.545300    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:26:25.545363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.545316    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245451d8-d8c5-4b7c-8445-3460460cf9fd-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:26:25.545363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:25.545330    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c46qw\" (UniqueName: \"kubernetes.io/projected/245451d8-d8c5-4b7c-8445-3460460cf9fd-kube-api-access-c46qw\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:26:26.274741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:26.274700    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l" event={"ID":"245451d8-d8c5-4b7c-8445-3460460cf9fd","Type":"ContainerDied","Data":"79527511fd5a5e9a6dfe68baf2a849c41ae8c9fcc98e253d4a5d133861d90478"}
Nov 02 03:26:26.274741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:26.274726    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"
Nov 02 03:26:26.275298 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:26.274783    4407 scope.go:122] "RemoveContainer" containerID="dd657d4a69ca5a7d61dd1c47996cbe7dda64910be045dbf684e4fc7c4c6f46dd"
Nov 02 03:26:26.378927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:26.378891    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:26.441472 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:26.441443    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"]
Nov 02 03:26:26.495681 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:26.495553    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b4a230-864bc5bc4d-c2d8l"]
Nov 02 03:26:26.672147 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:26.672062    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:26.916533 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:26.916500    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:27.077086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:27.077057    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:27.567840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:27.567804    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:28.352345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:28.352315    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="245451d8-d8c5-4b7c-8445-3460460cf9fd" path="/var/lib/kubelet/pods/245451d8-d8c5-4b7c-8445-3460460cf9fd/volumes"
Nov 02 03:26:32.076850 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:32.076822    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:33.276411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:33.276376    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:33.442444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:33.442414    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:33.612587 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:33.612507    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:35.188725 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:26:35.188686    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:26:37.114059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:37.114023    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:41.432762 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:41.432728    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:42.306682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:42.306642    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:43.245821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:43.245793    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:43.408869 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:43.408838    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:46.315057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:46.315026    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:46.464036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:46.464006    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:47.187320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:47.187287    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:51.976805 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:51.976777    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:56.771412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:56.771378    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:26:59.967554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:59.967521    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"]
Nov 02 03:26:59.968039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:26:59.967719    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy" containerID="cri-o://9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12" gracePeriod=30
Nov 02 03:27:01.561263 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:01.561227    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:03.888399 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:03.888363    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:04.045913 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:04.045881    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:04.901402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:04.901370    4407 patch_prober.go:29] interesting pod/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:27:04.901830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:04.901432    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:27:05.191099 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:05.191015    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:27:05.249039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.249017    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:27:05.342221 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342189    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-kube-api-access-4gxlj\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-kube-api-access-4gxlj\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342386 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342269    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-envoy\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342386 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342319    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-data\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342391    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-credential-socket\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342459    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-istio-token\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-istio-token\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342577 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342502    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1606ccff-f513-4084-920f-38d8b883f29e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1606ccff-f513-4084-920f-38d8b883f29e-istio-podinfo\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342577 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342535    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1606ccff-f513-4084-920f-38d8b883f29e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1606ccff-f513-4084-920f-38d8b883f29e-istiod-ca-cert\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342594    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-socket\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342665    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-data" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:05.342734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342706    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-certs\") pod \"1606ccff-f513-4084-920f-38d8b883f29e\" (UID: \"1606ccff-f513-4084-920f-38d8b883f29e\") "
Nov 02 03:27:05.342886 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342758    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-credential-socket" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:05.342942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342883    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-socket" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:05.343046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.343027    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-certs" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:05.343046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.342958    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1606ccff-f513-4084-920f-38d8b883f29e-istiod-ca-cert" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:05.343332 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.343312    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.343410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.343338    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.343410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.343353    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1606ccff-f513-4084-920f-38d8b883f29e-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.343410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.343366    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.343410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.343379    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.345008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.344974    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1606ccff-f513-4084-920f-38d8b883f29e-istio-podinfo" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:27:05.345121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.345094    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-kube-api-access-4gxlj" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "kube-api-access-4gxlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:05.345195 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.345180    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-istio-token" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:05.345236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.345189    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-envoy" pod "1606ccff-f513-4084-920f-38d8b883f29e" (UID: "1606ccff-f513-4084-920f-38d8b883f29e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:05.444854 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.444777    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.444854 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.444800    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1606ccff-f513-4084-920f-38d8b883f29e-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.444854 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.444815    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gxlj\" (UniqueName: \"kubernetes.io/projected/1606ccff-f513-4084-920f-38d8b883f29e-kube-api-access-4gxlj\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.444854 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.444829    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1606ccff-f513-4084-920f-38d8b883f29e-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:05.540082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.540049    4407 generic.go:356] "Generic (PLEG): container finished" podID="1606ccff-f513-4084-920f-38d8b883f29e" containerID="9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12" exitCode=0
Nov 02 03:27:05.540245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.540100    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"
Nov 02 03:27:05.540245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.540121    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" event={"ID":"1606ccff-f513-4084-920f-38d8b883f29e","Type":"ContainerDied","Data":"9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12"}
Nov 02 03:27:05.540245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.540174    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6" event={"ID":"1606ccff-f513-4084-920f-38d8b883f29e","Type":"ContainerDied","Data":"f7eade6daeace28efc37011e46ccc2dc5eb348db6eb892e82730c1ec94ef3a20"}
Nov 02 03:27:05.540245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.540194    4407 scope.go:122] "RemoveContainer" containerID="9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12"
Nov 02 03:27:05.550865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.550847    4407 scope.go:122] "RemoveContainer" containerID="9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12"
Nov 02 03:27:05.551130 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:05.551111    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12\": container with ID starting with 9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12 not found: ID does not exist" containerID="9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12"
Nov 02 03:27:05.551197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.551138    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12"} err="failed to get container status \"9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12\": rpc error: code = NotFound desc = could not find container \"9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12\": container with ID starting with 9d2315bd679a294152562cacdfb4379e3bdea519b520e622c2337401e660ee12 not found: ID does not exist"
Nov 02 03:27:05.708110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.708039    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"]
Nov 02 03:27:05.713663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.713611    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-bjxc6"]
Nov 02 03:27:05.892144 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:05.892116    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:06.352094 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:06.352066    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1606ccff-f513-4084-920f-38d8b883f29e" path="/var/lib/kubelet/pods/1606ccff-f513-4084-920f-38d8b883f29e/volumes"
Nov 02 03:27:07.358505 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:07.358471    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:07.582019 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:07.581984    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:07.737093 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:07.737013    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:08.259847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:08.259815    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:09.532057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.532022    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:09.690696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.690658    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:09.829947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.829872    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"]
Nov 02 03:27:09.830091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.830064    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy" containerID="cri-o://070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019" gracePeriod=30
Nov 02 03:27:09.849015 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.848989    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"]
Nov 02 03:27:09.849223 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.849201    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy" containerID="cri-o://171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a" gracePeriod=30
Nov 02 03:27:09.849311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.849202    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:09.873789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.873762    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"]
Nov 02 03:27:09.873962 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:09.873943    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy" containerID="cri-o://574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1" gracePeriod=30
Nov 02 03:27:10.593565 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:10.593533    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:11.218567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:11.218526    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:11.405850 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:11.405814    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:12.014832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.014797    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"]
Nov 02 03:27:12.015280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.014998    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h" podUID="667af376-6718-4ce6-8acf-2b3399c5282f" containerName="maas-api" containerID="cri-o://5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a" gracePeriod=30
Nov 02 03:27:12.142750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.142717    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:12.296223 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.296198    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:27:12.369046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.369016    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/667af376-6718-4ce6-8acf-2b3399c5282f-kube-api-access-rxgjj\" (UniqueName: \"kubernetes.io/projected/667af376-6718-4ce6-8acf-2b3399c5282f-kube-api-access-rxgjj\") pod \"667af376-6718-4ce6-8acf-2b3399c5282f\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") "
Nov 02 03:27:12.369231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.369073    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-metrics-certs\") pod \"667af376-6718-4ce6-8acf-2b3399c5282f\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") "
Nov 02 03:27:12.369231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.369183    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667af376-6718-4ce6-8acf-2b3399c5282f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667af376-6718-4ce6-8acf-2b3399c5282f-openshift-service-ca\") pod \"667af376-6718-4ce6-8acf-2b3399c5282f\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") "
Nov 02 03:27:12.369231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.369230    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls\") pod \"667af376-6718-4ce6-8acf-2b3399c5282f\" (UID: \"667af376-6718-4ce6-8acf-2b3399c5282f\") "
Nov 02 03:27:12.369548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.369524    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667af376-6718-4ce6-8acf-2b3399c5282f-openshift-service-ca" pod "667af376-6718-4ce6-8acf-2b3399c5282f" (UID: "667af376-6718-4ce6-8acf-2b3399c5282f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:12.369792 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.369744    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/667af376-6718-4ce6-8acf-2b3399c5282f-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:12.371306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.371274    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-metrics-certs" pod "667af376-6718-4ce6-8acf-2b3399c5282f" (UID: "667af376-6718-4ce6-8acf-2b3399c5282f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:27:12.371306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.371288    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667af376-6718-4ce6-8acf-2b3399c5282f-kube-api-access-rxgjj" pod "667af376-6718-4ce6-8acf-2b3399c5282f" (UID: "667af376-6718-4ce6-8acf-2b3399c5282f"). InnerVolumeSpecName "kube-api-access-rxgjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:12.371438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.371328    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls" pod "667af376-6718-4ce6-8acf-2b3399c5282f" (UID: "667af376-6718-4ce6-8acf-2b3399c5282f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:27:12.471603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.471566    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:12.471603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.471599    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxgjj\" (UniqueName: \"kubernetes.io/projected/667af376-6718-4ce6-8acf-2b3399c5282f-kube-api-access-rxgjj\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:12.471603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.471610    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/667af376-6718-4ce6-8acf-2b3399c5282f-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:12.595822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.595732    4407 generic.go:356] "Generic (PLEG): container finished" podID="667af376-6718-4ce6-8acf-2b3399c5282f" containerID="5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a" exitCode=0
Nov 02 03:27:12.595822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.595785    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"
Nov 02 03:27:12.596016 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.595826    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h" event={"ID":"667af376-6718-4ce6-8acf-2b3399c5282f","Type":"ContainerDied","Data":"5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a"}
Nov 02 03:27:12.596016 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.595926    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h" event={"ID":"667af376-6718-4ce6-8acf-2b3399c5282f","Type":"ContainerDied","Data":"91dee6e884612c3db2f79281cd285d723d4635fe712889c8bd2a02ccdda09d3f"}
Nov 02 03:27:12.596016 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.595960    4407 scope.go:122] "RemoveContainer" containerID="5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a"
Nov 02 03:27:12.608601 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.608585    4407 scope.go:122] "RemoveContainer" containerID="5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a"
Nov 02 03:27:12.608911 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:12.608890    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a\": container with ID starting with 5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a not found: ID does not exist" containerID="5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a"
Nov 02 03:27:12.608958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.608922    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a"} err="failed to get container status \"5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a\": rpc error: code = NotFound desc = could not find container \"5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a\": container with ID starting with 5c10ccc748d4a6bf366958cb903ffe2fbf2a75084036db5c58a3dd26bbe67f2a not found: ID does not exist"
Nov 02 03:27:12.764796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.764765    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"]
Nov 02 03:27:12.768345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:12.768320    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c9d1fc-6bd9c49d-qxq7h"]
Nov 02 03:27:13.139823 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.139793    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:27:13.185425 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.185398    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"]
Nov 02 03:27:13.185681 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.185636    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k" podUID="de6da90e-8860-48f3-beea-6b6819b6481d" containerName="maas-api" containerID="cri-o://0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631" gracePeriod=30
Nov 02 03:27:13.409707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.408596    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:13.499651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.499597    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:27:13.589158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.589126    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de6da90e-8860-48f3-beea-6b6819b6481d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6da90e-8860-48f3-beea-6b6819b6481d-openshift-service-ca\") pod \"de6da90e-8860-48f3-beea-6b6819b6481d\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") "
Nov 02 03:27:13.589839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.589818    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-metrics-certs\") pod \"de6da90e-8860-48f3-beea-6b6819b6481d\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") "
Nov 02 03:27:13.590070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.590055    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls\") pod \"de6da90e-8860-48f3-beea-6b6819b6481d\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") "
Nov 02 03:27:13.590283 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.590233    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de6da90e-8860-48f3-beea-6b6819b6481d-kube-api-access-vntfs\" (UniqueName: \"kubernetes.io/projected/de6da90e-8860-48f3-beea-6b6819b6481d-kube-api-access-vntfs\") pod \"de6da90e-8860-48f3-beea-6b6819b6481d\" (UID: \"de6da90e-8860-48f3-beea-6b6819b6481d\") "
Nov 02 03:27:13.591558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.589723    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de6da90e-8860-48f3-beea-6b6819b6481d-openshift-service-ca" pod "de6da90e-8860-48f3-beea-6b6819b6481d" (UID: "de6da90e-8860-48f3-beea-6b6819b6481d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:13.596906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.596504    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6da90e-8860-48f3-beea-6b6819b6481d-kube-api-access-vntfs" pod "de6da90e-8860-48f3-beea-6b6819b6481d" (UID: "de6da90e-8860-48f3-beea-6b6819b6481d"). InnerVolumeSpecName "kube-api-access-vntfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:13.599942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.599893    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls" pod "de6da90e-8860-48f3-beea-6b6819b6481d" (UID: "de6da90e-8860-48f3-beea-6b6819b6481d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:27:13.612765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.609765    4407 generic.go:356] "Generic (PLEG): container finished" podID="de6da90e-8860-48f3-beea-6b6819b6481d" containerID="0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631" exitCode=0
Nov 02 03:27:13.612765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.609817    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k" event={"ID":"de6da90e-8860-48f3-beea-6b6819b6481d","Type":"ContainerDied","Data":"0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631"}
Nov 02 03:27:13.612765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.609867    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k" event={"ID":"de6da90e-8860-48f3-beea-6b6819b6481d","Type":"ContainerDied","Data":"1b167635d299523c7d7d489d518f50ef60c333548b02d523add8b43455d2918b"}
Nov 02 03:27:13.612765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.609897    4407 scope.go:122] "RemoveContainer" containerID="0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631"
Nov 02 03:27:13.612765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.610037    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"
Nov 02 03:27:13.613835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.613790    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-metrics-certs" pod "de6da90e-8860-48f3-beea-6b6819b6481d" (UID: "de6da90e-8860-48f3-beea-6b6819b6481d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:27:13.640766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.640740    4407 scope.go:122] "RemoveContainer" containerID="0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631"
Nov 02 03:27:13.641131 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:13.641110    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631\": container with ID starting with 0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631 not found: ID does not exist" containerID="0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631"
Nov 02 03:27:13.641185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.641139    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631"} err="failed to get container status \"0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631\": rpc error: code = NotFound desc = could not find container \"0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631\": container with ID starting with 0f82c62add54d383e56df40f714f826129c37f04f4606511882036c1fc900631 not found: ID does not exist"
Nov 02 03:27:13.692788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.692689    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6da90e-8860-48f3-beea-6b6819b6481d-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:13.692788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.692724    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:13.692788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.692734    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6da90e-8860-48f3-beea-6b6819b6481d-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:13.692788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.692743    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vntfs\" (UniqueName: \"kubernetes.io/projected/de6da90e-8860-48f3-beea-6b6819b6481d-kube-api-access-vntfs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:13.797678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:13.792524    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:14.030353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:14.030315    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:14.032479 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:14.032458    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"]
Nov 02 03:27:14.038082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:14.038058    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-018cfa-5bb997764b-9p77k"]
Nov 02 03:27:14.385239 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:14.385155    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="667af376-6718-4ce6-8acf-2b3399c5282f" path="/var/lib/kubelet/pods/667af376-6718-4ce6-8acf-2b3399c5282f/volumes"
Nov 02 03:27:14.391994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:14.391966    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de6da90e-8860-48f3-beea-6b6819b6481d" path="/var/lib/kubelet/pods/de6da90e-8860-48f3-beea-6b6819b6481d/volumes"
Nov 02 03:27:14.618348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:14.618307    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:14.856830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:14.856801    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:15.028201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.028168    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:15.155068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.155037    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"]
Nov 02 03:27:15.155767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.155292    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy" containerID="cri-o://612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75" gracePeriod=30
Nov 02 03:27:15.184176 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.184145    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:27:15.186284 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.186260    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:27:15.199864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.199825    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:15.214282 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214103    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-data\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.214282 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214213    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-token\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-token\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.214282 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214248    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-credential-socket\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.214282 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214289    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istiod-ca-cert\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214332    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-certs\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214366    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-socket\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214397    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-socket\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214404    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-data" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214475    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-envoy\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214504    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-envoy\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214542    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-kube-api-access-nn9lg\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-kube-api-access-nn9lg\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214576    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-data\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.214767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214617    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-certs" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.215121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.214927    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-socket" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.215121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215022    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-socket" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.215121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215109    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istiod-ca-cert" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:15.215323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215297    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-data" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.215650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215600    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-certs\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.215650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215608    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-credential-socket" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.215797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215687    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-kube-api-access-q8jsz\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-kube-api-access-q8jsz\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.215797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215723    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-podinfo\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.215797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215760    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istiod-ca-cert\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.216916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.215792    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-credential-socket\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.216916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.216096    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-token\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-token\") pod \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\" (UID: \"3eb62fe9-84bd-4362-9bca-d179e60db6d0\") "
Nov 02 03:27:15.216916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.216188    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-podinfo\") pod \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\" (UID: \"31c79a00-a599-47d6-90f4-6e474cdf4d8a\") "
Nov 02 03:27:15.216916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.216371    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-credential-socket" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.216916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.216416    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istiod-ca-cert" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:15.216916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.216641    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-certs" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217050    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217072    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217087    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217101    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217115    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217129    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217143    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217160    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217173    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.217336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217188    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.218262 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.217734    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-envoy" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.219972 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.219943    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-envoy" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.220152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.220123    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-kube-api-access-nn9lg" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "kube-api-access-nn9lg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:15.220222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.220169    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-token" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:15.220699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.220678    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-kube-api-access-q8jsz" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "kube-api-access-q8jsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:15.221034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.221011    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-podinfo" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:27:15.221155 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.221032    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-podinfo" pod "31c79a00-a599-47d6-90f4-6e474cdf4d8a" (UID: "31c79a00-a599-47d6-90f4-6e474cdf4d8a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:27:15.221450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.221432    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-token" pod "3eb62fe9-84bd-4362-9bca-d179e60db6d0" (UID: "3eb62fe9-84bd-4362-9bca-d179e60db6d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:15.237438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.237419    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362402    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-credential-socket\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362486    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-certs\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362523    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-envoy\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362578    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-socket\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362611    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-istio-token\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-istio-token\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362674    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-data\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362713    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/94c76785-3504-4459-9980-245d9faf185d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94c76785-3504-4459-9980-245d9faf185d-istio-podinfo\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362752    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-kube-api-access-4tjsg\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-kube-api-access-4tjsg\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.362853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.362816    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94c76785-3504-4459-9980-245d9faf185d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94c76785-3504-4459-9980-245d9faf185d-istiod-ca-cert\") pod \"94c76785-3504-4459-9980-245d9faf185d\" (UID: \"94c76785-3504-4459-9980-245d9faf185d\") "
Nov 02 03:27:15.363870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.363312    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.363870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.363334    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.363870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.363350    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.363870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.363367    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn9lg\" (UniqueName: \"kubernetes.io/projected/31c79a00-a599-47d6-90f4-6e474cdf4d8a-kube-api-access-nn9lg\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.363870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.363381    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8jsz\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-kube-api-access-q8jsz\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.363870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.363397    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.363870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.363409    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb62fe9-84bd-4362-9bca-d179e60db6d0-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.363870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.363422    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c79a00-a599-47d6-90f4-6e474cdf4d8a-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.366383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.365381    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-data" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.366383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.365707    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-credential-socket" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.366383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.365770    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-socket" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.366383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.365952    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-certs" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.370940 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.370819    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:15.371559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.371380    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94c76785-3504-4459-9980-245d9faf185d-istiod-ca-cert" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:15.387999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.387839    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-envoy" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:15.388497 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.388167    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94c76785-3504-4459-9980-245d9faf185d-istio-podinfo" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:27:15.391235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.391172    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-istio-token" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:15.391819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.391789    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-kube-api-access-4tjsg" pod "94c76785-3504-4459-9980-245d9faf185d" (UID: "94c76785-3504-4459-9980-245d9faf185d"). InnerVolumeSpecName "kube-api-access-4tjsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:15.465264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465233    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.465264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465265    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.465488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465280    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.465488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465295    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.465488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465309    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.465488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465323    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94c76785-3504-4459-9980-245d9faf185d-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.465488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465337    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94c76785-3504-4459-9980-245d9faf185d-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.465488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465351    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tjsg\" (UniqueName: \"kubernetes.io/projected/94c76785-3504-4459-9980-245d9faf185d-kube-api-access-4tjsg\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.465488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.465366    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94c76785-3504-4459-9980-245d9faf185d-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:15.544165 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.544034    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:15.626486 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.626452    4407 generic.go:356] "Generic (PLEG): container finished" podID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerID="070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019" exitCode=0
Nov 02 03:27:15.626703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.626510    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"
Nov 02 03:27:15.626703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.626539    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" event={"ID":"31c79a00-a599-47d6-90f4-6e474cdf4d8a","Type":"ContainerDied","Data":"070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019"}
Nov 02 03:27:15.626703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.626608    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k" event={"ID":"31c79a00-a599-47d6-90f4-6e474cdf4d8a","Type":"ContainerDied","Data":"9f2f878b27d56c136a234244cacb2664e7c8c7b4328eabaa454beadc921d9008"}
Nov 02 03:27:15.626703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.626658    4407 scope.go:122] "RemoveContainer" containerID="070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019"
Nov 02 03:27:15.628078 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.628054    4407 generic.go:356] "Generic (PLEG): container finished" podID="94c76785-3504-4459-9980-245d9faf185d" containerID="574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1" exitCode=0
Nov 02 03:27:15.628369 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.628100    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"
Nov 02 03:27:15.628369 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.628149    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" event={"ID":"94c76785-3504-4459-9980-245d9faf185d","Type":"ContainerDied","Data":"574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1"}
Nov 02 03:27:15.628369 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.628192    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8" event={"ID":"94c76785-3504-4459-9980-245d9faf185d","Type":"ContainerDied","Data":"f2dd590a39d4148867d517d90de2c189818eb49755636dc65cc1f4aa79f3d832"}
Nov 02 03:27:15.629701 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.629683    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"
Nov 02 03:27:15.629701 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.629611    4407 generic.go:356] "Generic (PLEG): container finished" podID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerID="171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a" exitCode=0
Nov 02 03:27:15.629869 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.629681    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" event={"ID":"3eb62fe9-84bd-4362-9bca-d179e60db6d0","Type":"ContainerDied","Data":"171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a"}
Nov 02 03:27:15.629869 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.629808    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7" event={"ID":"3eb62fe9-84bd-4362-9bca-d179e60db6d0","Type":"ContainerDied","Data":"bcfe2823d7d3df09a3cb9f9b187f33b0926684395c3127811bca843de1cf21ae"}
Nov 02 03:27:15.641299 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.640737    4407 scope.go:122] "RemoveContainer" containerID="94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146"
Nov 02 03:27:15.658045 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.658021    4407 scope.go:122] "RemoveContainer" containerID="070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019"
Nov 02 03:27:15.658504 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:15.658483    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019\": container with ID starting with 070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019 not found: ID does not exist" containerID="070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019"
Nov 02 03:27:15.658593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.658518    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019"} err="failed to get container status \"070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019\": rpc error: code = NotFound desc = could not find container \"070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019\": container with ID starting with 070bd3403134b4118ada4914d0325b07dfddd30946e9ce6df087283188eaf019 not found: ID does not exist"
Nov 02 03:27:15.658593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.658538    4407 scope.go:122] "RemoveContainer" containerID="94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146"
Nov 02 03:27:15.659241 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:15.659210    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146\": container with ID starting with 94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146 not found: ID does not exist" containerID="94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146"
Nov 02 03:27:15.659340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.659249    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146"} err="failed to get container status \"94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146\": rpc error: code = NotFound desc = could not find container \"94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146\": container with ID starting with 94829b74741b95326627cd72b750d4bc63504fbda3ea829de8efa8e2454b6146 not found: ID does not exist"
Nov 02 03:27:15.659340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.659268    4407 scope.go:122] "RemoveContainer" containerID="574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1"
Nov 02 03:27:15.670546 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.670520    4407 scope.go:122] "RemoveContainer" containerID="574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1"
Nov 02 03:27:15.671206 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:15.670945    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1\": container with ID starting with 574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1 not found: ID does not exist" containerID="574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1"
Nov 02 03:27:15.671206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.670988    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1"} err="failed to get container status \"574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1\": rpc error: code = NotFound desc = could not find container \"574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1\": container with ID starting with 574e734d464cd522568a164dabdbda6f51d8fd1151300fc2c4fdb12278a649b1 not found: ID does not exist"
Nov 02 03:27:15.671206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.671012    4407 scope.go:122] "RemoveContainer" containerID="171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a"
Nov 02 03:27:15.681220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.681198    4407 scope.go:122] "RemoveContainer" containerID="171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a"
Nov 02 03:27:15.681525 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:15.681500    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a\": container with ID starting with 171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a not found: ID does not exist" containerID="171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a"
Nov 02 03:27:15.681652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.681535    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a"} err="failed to get container status \"171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a\": rpc error: code = NotFound desc = could not find container \"171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a\": container with ID starting with 171a3234b012f960be1b1e70f043135a3bc8672e5a6650d83475281f6b40b92a not found: ID does not exist"
Nov 02 03:27:15.917455 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.917377    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"]
Nov 02 03:27:15.922787 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:15.922764    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-k649k"]
Nov 02 03:27:16.026221 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.026183    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:16.037568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.037538    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"]
Nov 02 03:27:16.037807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.037793    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-tcdp8"]
Nov 02 03:27:16.151359 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.151331    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"]
Nov 02 03:27:16.156080 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.156054    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d5dcca72-openshift-default-7c7c99b4c7-h2hj7"]
Nov 02 03:27:16.197606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.197531    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:16.352733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.352703    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" path="/var/lib/kubelet/pods/31c79a00-a599-47d6-90f4-6e474cdf4d8a/volumes"
Nov 02 03:27:16.353247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.353233    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" path="/var/lib/kubelet/pods/3eb62fe9-84bd-4362-9bca-d179e60db6d0/volumes"
Nov 02 03:27:16.353675 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.353662    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94c76785-3504-4459-9980-245d9faf185d" path="/var/lib/kubelet/pods/94c76785-3504-4459-9980-245d9faf185d/volumes"
Nov 02 03:27:16.370828 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:16.370802    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:19.286395 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:19.286306    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:19.454984 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:19.454947    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:20.433349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.433291    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:27:20.531007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.530975    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-kube-api-access-gdzmg\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-kube-api-access-gdzmg\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531213 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531075    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-podinfo\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531213 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531110    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-socket\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531213 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531127    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-envoy\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531213 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531165    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-token\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-token\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531248    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-certs\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531281    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istiod-ca-cert\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531316    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-data\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531371    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-credential-socket\") pod \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\" (UID: \"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9\") "
Nov 02 03:27:20.531652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531422    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-socket" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:20.531719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531676    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-data" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:20.531798 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531769    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istiod-ca-cert" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:20.531895 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.531873    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-certs" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:20.532086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.532046    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-credential-socket" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:20.532177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.532147    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.532242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.532170    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.532242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.532194    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.532242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.532208    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.533538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.533515    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-envoy" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:20.533849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.533833    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-podinfo" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:27:20.533931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.533870    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-kube-api-access-gdzmg" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "kube-api-access-gdzmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:20.534276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.534257    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-token" pod "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" (UID: "b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:20.542771 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.542745    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:20.633357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.633310    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.633357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.633346    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdzmg\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-kube-api-access-gdzmg\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.633357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.633361    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.633616 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.633373    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.633616 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.633386    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:20.664806 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.664774    4407 generic.go:356] "Generic (PLEG): container finished" podID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerID="612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75" exitCode=0
Nov 02 03:27:20.664969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.664828    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" event={"ID":"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9","Type":"ContainerDied","Data":"612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75"}
Nov 02 03:27:20.664969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.664838    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"
Nov 02 03:27:20.664969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.664939    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz" event={"ID":"b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9","Type":"ContainerDied","Data":"87ad3e473ab9c8d65dfd367ff7f5d2f999f160a48d74520a3ca67fdc561d67a3"}
Nov 02 03:27:20.665083 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.664975    4407 scope.go:122] "RemoveContainer" containerID="612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75"
Nov 02 03:27:20.675664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.675613    4407 scope.go:122] "RemoveContainer" containerID="6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709"
Nov 02 03:27:20.684436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.684417    4407 scope.go:122] "RemoveContainer" containerID="612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75"
Nov 02 03:27:20.684716 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:20.684697    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75\": container with ID starting with 612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75 not found: ID does not exist" containerID="612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75"
Nov 02 03:27:20.684784 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.684728    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75"} err="failed to get container status \"612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75\": rpc error: code = NotFound desc = could not find container \"612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75\": container with ID starting with 612b8ed38ad953d3f7b90fc58cd2e3c9c7d85dc6cb2dc449adc0c230d9e36e75 not found: ID does not exist"
Nov 02 03:27:20.684784 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.684746    4407 scope.go:122] "RemoveContainer" containerID="6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709"
Nov 02 03:27:20.684974 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:20.684958    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709\": container with ID starting with 6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709 not found: ID does not exist" containerID="6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709"
Nov 02 03:27:20.685016 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.684978    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709"} err="failed to get container status \"6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709\": rpc error: code = NotFound desc = could not find container \"6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709\": container with ID starting with 6272b9cd1283f833f36cf2992810a72a0b55db6fb99ba6fee5449918e6a10709 not found: ID does not exist"
Nov 02 03:27:20.705291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.705272    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:20.832405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.832373    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"]
Nov 02 03:27:20.836760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.836733    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d6544b-openshift-default-799cbf6845-xw2gz"]
Nov 02 03:27:20.864947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:20.864911    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:22.302663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:22.302609    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:22.357168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:22.357130    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" path="/var/lib/kubelet/pods/b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9/volumes"
Nov 02 03:27:22.463795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:22.463769    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:22.619708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:22.619612    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:25.586959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:25.586931    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:30.720891 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:30.720830    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:33.525769 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:33.525739    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:33.707664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:33.707609    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:35.189714 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:35.189670    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:27:35.769050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:35.768999    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:39.185223 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:39.185193    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:27:40.680463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:40.680431    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:41.650649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:41.650175    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"]
Nov 02 03:27:41.650874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:41.650771    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy" containerID="cri-o://99b1a06056f8aaa2cb64ad48cf880bdb63dc6f0b7e354258677a15436da3fa77" gracePeriod=30
Nov 02 03:27:41.778203 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:41.778176    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:44.132277 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:44.132239    4407 patch_prober.go:29] interesting pod/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:27:44.132733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:44.132307    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:27:44.735962 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:44.735930    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"]
Nov 02 03:27:44.736145 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:44.736117    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567" podUID="1dc888ac-f7c8-483c-bbda-75ec89265b77" containerName="maas-api" containerID="cri-o://d9a7bbffd359076b8659581daa08eb90c16167d1de1e6f240d94b9c79f1c62c9" gracePeriod=30
Nov 02 03:27:44.822384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:44.822349    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:44.893474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:44.893430    4407 generic.go:356] "Generic (PLEG): container finished" podID="1dc888ac-f7c8-483c-bbda-75ec89265b77" containerID="d9a7bbffd359076b8659581daa08eb90c16167d1de1e6f240d94b9c79f1c62c9" exitCode=0
Nov 02 03:27:44.893687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:44.893496    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567" event={"ID":"1dc888ac-f7c8-483c-bbda-75ec89265b77","Type":"ContainerDied","Data":"d9a7bbffd359076b8659581daa08eb90c16167d1de1e6f240d94b9c79f1c62c9"}
Nov 02 03:27:45.015461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.015429    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:27:45.108614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.108578    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls\") pod \"1dc888ac-f7c8-483c-bbda-75ec89265b77\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") "
Nov 02 03:27:45.108834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.108738    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1dc888ac-f7c8-483c-bbda-75ec89265b77-kube-api-access-krblb\" (UniqueName: \"kubernetes.io/projected/1dc888ac-f7c8-483c-bbda-75ec89265b77-kube-api-access-krblb\") pod \"1dc888ac-f7c8-483c-bbda-75ec89265b77\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") "
Nov 02 03:27:45.108834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.108781    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-metrics-certs\") pod \"1dc888ac-f7c8-483c-bbda-75ec89265b77\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") "
Nov 02 03:27:45.108834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.108809    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1dc888ac-f7c8-483c-bbda-75ec89265b77-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dc888ac-f7c8-483c-bbda-75ec89265b77-openshift-service-ca\") pod \"1dc888ac-f7c8-483c-bbda-75ec89265b77\" (UID: \"1dc888ac-f7c8-483c-bbda-75ec89265b77\") "
Nov 02 03:27:45.109210 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.109178    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dc888ac-f7c8-483c-bbda-75ec89265b77-openshift-service-ca" pod "1dc888ac-f7c8-483c-bbda-75ec89265b77" (UID: "1dc888ac-f7c8-483c-bbda-75ec89265b77"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:45.111045 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.111012    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls" pod "1dc888ac-f7c8-483c-bbda-75ec89265b77" (UID: "1dc888ac-f7c8-483c-bbda-75ec89265b77"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:27:45.111150 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.111073    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-metrics-certs" pod "1dc888ac-f7c8-483c-bbda-75ec89265b77" (UID: "1dc888ac-f7c8-483c-bbda-75ec89265b77"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:27:45.111150 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.111105    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc888ac-f7c8-483c-bbda-75ec89265b77-kube-api-access-krblb" pod "1dc888ac-f7c8-483c-bbda-75ec89265b77" (UID: "1dc888ac-f7c8-483c-bbda-75ec89265b77"). InnerVolumeSpecName "kube-api-access-krblb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:45.174701 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.174663    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:45.210609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.210561    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:45.210609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.210595    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krblb\" (UniqueName: \"kubernetes.io/projected/1dc888ac-f7c8-483c-bbda-75ec89265b77-kube-api-access-krblb\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:45.210609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.210605    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc888ac-f7c8-483c-bbda-75ec89265b77-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:45.210609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.210615    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dc888ac-f7c8-483c-bbda-75ec89265b77-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:45.900178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.900152    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"
Nov 02 03:27:45.900357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.900152    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567" event={"ID":"1dc888ac-f7c8-483c-bbda-75ec89265b77","Type":"ContainerDied","Data":"28f6a22e3811e704b20ba75aaad4a5db54a540a12fa1f2a22ee0c9b260a59303"}
Nov 02 03:27:45.900357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:45.900286    4407 scope.go:122] "RemoveContainer" containerID="d9a7bbffd359076b8659581daa08eb90c16167d1de1e6f240d94b9c79f1c62c9"
Nov 02 03:27:46.071584 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.071545    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"]
Nov 02 03:27:46.080667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.080641    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a27da5-66946cdd4c-pm567"]
Nov 02 03:27:46.209173 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.209097    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:46.356255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.356221    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1dc888ac-f7c8-483c-bbda-75ec89265b77" path="/var/lib/kubelet/pods/1dc888ac-f7c8-483c-bbda-75ec89265b77/volumes"
Nov 02 03:27:46.392489 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.392459    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:46.547172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.547145    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:46.702682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.702658    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:46.871887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.871848    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:46.910461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.910434    4407 generic.go:356] "Generic (PLEG): container finished" podID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerID="99b1a06056f8aaa2cb64ad48cf880bdb63dc6f0b7e354258677a15436da3fa77" exitCode=0
Nov 02 03:27:46.910604 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.910497    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" event={"ID":"a3b57248-092f-4876-b4f2-ce7bfb44fe7b","Type":"ContainerDied","Data":"99b1a06056f8aaa2cb64ad48cf880bdb63dc6f0b7e354258677a15436da3fa77"}
Nov 02 03:27:46.910604 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.910553    4407 scope.go:122] "RemoveContainer" containerID="b189826f864ccecb8abfdc8434084f12d388caada9077b7dec3e9ee6683778a3"
Nov 02 03:27:46.934365 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:46.934344    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:27:47.034952 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.034915    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istiod-ca-cert\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035027    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-kube-api-access-29tcd\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-kube-api-access-29tcd\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035057    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-credential-socket\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035095    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-socket\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035145    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-certs\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035177    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-envoy\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035210    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-token\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-token\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035267    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-podinfo\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035354    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-data\") pod \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\" (UID: \"a3b57248-092f-4876-b4f2-ce7bfb44fe7b\") "
Nov 02 03:27:47.035526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035350    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-credential-socket" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:47.035526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035276    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istiod-ca-cert" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:47.035526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035461    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-certs" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:47.035768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035613    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-data" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:47.035768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.035756    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-socket" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:47.036132 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.036064    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.036132 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.036090    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.036132 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.036106    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.036132 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.036121    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.036132 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.036135    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.037457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.037428    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-podinfo" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:27:47.037585 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.037489    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-kube-api-access-29tcd" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "kube-api-access-29tcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:47.037585 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.037518    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-token" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:47.037699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.037584    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-envoy" pod "a3b57248-092f-4876-b4f2-ce7bfb44fe7b" (UID: "a3b57248-092f-4876-b4f2-ce7bfb44fe7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:47.137266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.137178    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29tcd\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-kube-api-access-29tcd\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.137266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.137214    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.137266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.137223    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.137266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.137232    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b57248-092f-4876-b4f2-ce7bfb44fe7b-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:47.334613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.334579    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:47.917023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.916995    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"
Nov 02 03:27:47.917193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.917032    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2" event={"ID":"a3b57248-092f-4876-b4f2-ce7bfb44fe7b","Type":"ContainerDied","Data":"6de1cb43a8b92c94f8560c0ee4441ed706d49509e7358709033a35eb913e83e8"}
Nov 02 03:27:47.917193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:47.917089    4407 scope.go:122] "RemoveContainer" containerID="99b1a06056f8aaa2cb64ad48cf880bdb63dc6f0b7e354258677a15436da3fa77"
Nov 02 03:27:48.205491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:48.205394    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"]
Nov 02 03:27:48.211243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:48.211217    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-cszr2"]
Nov 02 03:27:48.353007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:48.352973    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" path="/var/lib/kubelet/pods/a3b57248-092f-4876-b4f2-ce7bfb44fe7b/volumes"
Nov 02 03:27:50.348481 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:50.348426    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:50.519420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:50.519384    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:50.679536 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:50.679421    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:51.850704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:51.850674    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:52.016031 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:52.015968    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:52.172080 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:52.171968    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:53.205615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:53.205587    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:53.352154 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:53.352101    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:53.489820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:53.489741    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"]
Nov 02 03:27:53.489959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:53.489943    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy" containerID="cri-o://d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e" gracePeriod=30
Nov 02 03:27:53.520846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:53.520818    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:54.424224 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:54.424194    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:54.915658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:54.915590    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:55.078819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.078784    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"]
Nov 02 03:27:55.079554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079522    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy"
Nov 02 03:27:55.079554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079549    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079564    4407 cpu_manager.go:414] "containerMap: removing container" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079572    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079588    4407 cpu_manager.go:414] "containerMap: removing container" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079595    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079606    4407 cpu_manager.go:414] "containerMap: removing container" podUID="de6da90e-8860-48f3-beea-6b6819b6481d" containerName="maas-api"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079613    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de6da90e-8860-48f3-beea-6b6819b6481d" containerName="maas-api"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079654    4407 cpu_manager.go:414] "containerMap: removing container" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079661    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079674    4407 cpu_manager.go:414] "containerMap: removing container" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079679    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079689    4407 cpu_manager.go:414] "containerMap: removing container" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079695    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079711    4407 cpu_manager.go:414] "containerMap: removing container" podUID="60e5f97c-02aa-4d35-abed-284be1654094" containerName="maas-api"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079718    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60e5f97c-02aa-4d35-abed-284be1654094" containerName="maas-api"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079735    4407 cpu_manager.go:414] "containerMap: removing container" podUID="245451d8-d8c5-4b7c-8445-3460460cf9fd" containerName="maas-api"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079742    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="245451d8-d8c5-4b7c-8445-3460460cf9fd" containerName="maas-api"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079753    4407 cpu_manager.go:414] "containerMap: removing container" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079760    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079769    4407 cpu_manager.go:414] "containerMap: removing container" podUID="667af376-6718-4ce6-8acf-2b3399c5282f" containerName="maas-api"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079776    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667af376-6718-4ce6-8acf-2b3399c5282f" containerName="maas-api"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079787    4407 cpu_manager.go:414] "containerMap: removing container" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079794    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079807    4407 cpu_manager.go:414] "containerMap: removing container" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy"
Nov 02 03:27:55.079810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079814    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079830    4407 cpu_manager.go:414] "containerMap: removing container" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079837    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079848    4407 cpu_manager.go:414] "containerMap: removing container" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079855    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079867    4407 cpu_manager.go:414] "containerMap: removing container" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079873    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079888    4407 cpu_manager.go:414] "containerMap: removing container" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079896    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079915    4407 cpu_manager.go:414] "containerMap: removing container" podUID="1dc888ac-f7c8-483c-bbda-75ec89265b77" containerName="maas-api"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079923    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1dc888ac-f7c8-483c-bbda-75ec89265b77" containerName="maas-api"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079943    4407 cpu_manager.go:414] "containerMap: removing container" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.079950    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080086    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="1606ccff-f513-4084-920f-38d8b883f29e" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080101    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="1dc888ac-f7c8-483c-bbda-75ec89265b77" containerName="maas-api"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080112    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080123    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e8e57a2-8970-4f43-ad20-ed3a3642268c" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080137    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080146    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080156    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4cd0ea3-6d06-47f1-98a2-a4f94645a44a" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080165    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="de6da90e-8860-48f3-beea-6b6819b6481d" containerName="maas-api"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080174    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080183    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080191    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="c17d8e32-b682-4c2e-8d56-b5aecc10042b" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080200    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="60e5f97c-02aa-4d35-abed-284be1654094" containerName="maas-api"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080210    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="245451d8-d8c5-4b7c-8445-3460460cf9fd" containerName="maas-api"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080217    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080225    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="667af376-6718-4ce6-8acf-2b3399c5282f" containerName="maas-api"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080234    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="94c76785-3504-4459-9980-245d9faf185d" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080243    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="31c79a00-a599-47d6-90f4-6e474cdf4d8a" containerName="istio-proxy"
Nov 02 03:27:55.080999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.080252    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="3eb62fe9-84bd-4362-9bca-d179e60db6d0" containerName="istio-proxy"
Nov 02 03:27:55.086072 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.086046    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.089147 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.089120    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:55.139311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.139274    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-dd87f0-openshift-default-dockercfg-qmjlg\""
Nov 02 03:27:55.143914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.143880    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-data\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.144084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.143926    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-podinfo\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.144084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.144047    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-envoy\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.144084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.144078    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-socket\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.144251 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.144115    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-certs\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.144251 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.144240    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-credential-socket\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.144347 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.144278    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8w8l\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-kube-api-access-v8w8l\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.144383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.144370    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-token\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.144499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.144482    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istiod-ca-cert\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.192953 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.192855    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"]
Nov 02 03:27:55.245242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245211    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8w8l\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-kube-api-access-v8w8l\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.245434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245264    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-token\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.245517 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245426    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istiod-ca-cert\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.245579 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245536    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-data\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.245669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245580    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-podinfo\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.245669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245657    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-envoy\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.245786 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245691    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-socket\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.245786 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245728    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-certs\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.245913 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.245893    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-credential-socket\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.246124 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.246092    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-data\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.246244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.246187    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-socket\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.246244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.246233    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-credential-socket\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.246368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.246334    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-certs\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.246487 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.246408    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istiod-ca-cert\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.248189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.248167    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-envoy\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.248365 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.248349    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-podinfo\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.264152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.264108    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:55.311666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.311606    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-token\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.311829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.311761    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8w8l\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-kube-api-access-v8w8l\") pod \"e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") " pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.428783 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.428548    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:55.436469 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.436068    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:55.883259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.883218    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"]
Nov 02 03:27:55.922363 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:27:55.922336    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c937a7_e688_49e8_ab6e_6726e3afa7a9.slice/crio-728a6677d9e314887bbe22d7b03ebec0833a8117ccc43953ea66095adafbcc5d WatchSource:0}: Error finding container 728a6677d9e314887bbe22d7b03ebec0833a8117ccc43953ea66095adafbcc5d: Status 404 returned error can't find the container with id 728a6677d9e314887bbe22d7b03ebec0833a8117ccc43953ea66095adafbcc5d
Nov 02 03:27:55.924750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.924717    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:27:55.924820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.924790    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:27:55.924864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.924824    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:27:55.967114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:55.967087    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" event={"ID":"06c937a7-e688-49e8-ab6e-6726e3afa7a9","Type":"ContainerStarted","Data":"728a6677d9e314887bbe22d7b03ebec0833a8117ccc43953ea66095adafbcc5d"}
Nov 02 03:27:56.975265 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:56.975221    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" event={"ID":"06c937a7-e688-49e8-ab6e-6726e3afa7a9","Type":"ContainerStarted","Data":"09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701"}
Nov 02 03:27:57.143263 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.143211    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" podStartSLOduration=2.143197644 podStartE2EDuration="2.143197644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:27:57.142809523 +0000 UTC m=+2420.938158090" watchObservedRunningTime="2026-11-02 03:27:57.143197644 +0000 UTC m=+2420.938546208"
Nov 02 03:27:57.196248 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.196215    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"]
Nov 02 03:27:57.196758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.196744    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy"
Nov 02 03:27:57.196807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.196759    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3b57248-092f-4876-b4f2-ce7bfb44fe7b" containerName="istio-proxy"
Nov 02 03:27:57.196807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.196769    4407 cpu_manager.go:414] "containerMap: removing container" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy"
Nov 02 03:27:57.196807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.196773    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy"
Nov 02 03:27:57.196901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.196846    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2ffaa24-4dd7-44cc-8a50-9b7bd66145a9" containerName="istio-proxy"
Nov 02 03:27:57.196901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.196859    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="c42b386b-9a17-4c21-b5f4-7cc0934d358a" containerName="istio-proxy"
Nov 02 03:27:57.196901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.196868    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="b95248c7-bf6d-4b12-9fa4-9c0b1fc1537b" containerName="istio-proxy"
Nov 02 03:27:57.200249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.200233    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.290469 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.290416    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.290683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.290577    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.290683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.290652    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.290821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.290795    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.290871 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.290854    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.290931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.290914    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/300055ac-05f3-4374-ab56-6d62f61699b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.290972 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.290944    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8npnh\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-kube-api-access-8npnh\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.291143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.291125    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/300055ac-05f3-4374-ab56-6d62f61699b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.291185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.291161    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.361951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.361903    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:27:57.392453 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.392412    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/300055ac-05f3-4374-ab56-6d62f61699b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.392665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.392465    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.392665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.392488    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.392665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.392543    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.392665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.392580    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.393245    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.393307    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/300055ac-05f3-4374-ab56-6d62f61699b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.393962    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.394021    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.394053    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/300055ac-05f3-4374-ab56-6d62f61699b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.394087    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8npnh\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-kube-api-access-8npnh\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.394812    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.395132    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.395162    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.395944    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.396831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.396512    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"]
Nov 02 03:27:57.397826 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.397320    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/300055ac-05f3-4374-ab56-6d62f61699b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.437188 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.437102    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:57.442874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.442851    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:57.452458 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.452429    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8npnh\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-kube-api-access-8npnh\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.452662 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.452441    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-q9rnh\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.514457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.514427    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:57.939461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.939429    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"]
Nov 02 03:27:57.940287 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:27:57.940263    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod300055ac_05f3_4374_ab56_6d62f61699b5.slice/crio-f92e18a2b1dcf0439e3406400eb3dd74a6379db507f0500223c857b6de8e6b00 WatchSource:0}: Error finding container f92e18a2b1dcf0439e3406400eb3dd74a6379db507f0500223c857b6de8e6b00: Status 404 returned error can't find the container with id f92e18a2b1dcf0439e3406400eb3dd74a6379db507f0500223c857b6de8e6b00
Nov 02 03:27:57.942903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.942869    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:27:57.942990 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.942933    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:27:57.942990 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.942967    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:27:57.983944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.983897    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" event={"ID":"300055ac-05f3-4374-ab56-6d62f61699b5","Type":"ContainerStarted","Data":"f92e18a2b1dcf0439e3406400eb3dd74a6379db507f0500223c857b6de8e6b00"}
Nov 02 03:27:57.984340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.984240    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:57.985854 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:57.985834    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:27:58.779353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.779329    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:27:58.922476 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922446    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-kube-api-access-j6fxw\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-kube-api-access-j6fxw\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.922688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922517    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-credential-socket\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.922688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922565    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-socket\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.922688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922633    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-podinfo\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.922878 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922754    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-token\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-token\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.922878 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922839    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-certs\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.922963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922877    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-data\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.922963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922878    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-credential-socket" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:58.922963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922919    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-envoy\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.923055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.922976    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6e61c64-563d-4c06-b771-89b52ff8191b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e61c64-563d-4c06-b771-89b52ff8191b-istiod-ca-cert\") pod \"e6e61c64-563d-4c06-b771-89b52ff8191b\" (UID: \"e6e61c64-563d-4c06-b771-89b52ff8191b\") "
Nov 02 03:27:58.923164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.923142    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-data" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:58.923447 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.923423    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6e61c64-563d-4c06-b771-89b52ff8191b-istiod-ca-cert" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:27:58.923603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.923578    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:58.923865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.923607    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:58.923865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.923642    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e61c64-563d-4c06-b771-89b52ff8191b-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:58.923865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.923821    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-socket" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:58.924026 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.923860    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-certs" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:58.925321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.925285    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-envoy" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:27:58.925559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.925540    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-kube-api-access-j6fxw" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "kube-api-access-j6fxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:58.925735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.925709    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-token" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:27:58.925811 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:58.925717    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-podinfo" pod "e6e61c64-563d-4c06-b771-89b52ff8191b" (UID: "e6e61c64-563d-4c06-b771-89b52ff8191b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:27:59.012998 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.012905    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" event={"ID":"300055ac-05f3-4374-ab56-6d62f61699b5","Type":"ContainerStarted","Data":"3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6"}
Nov 02 03:27:59.014800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.014771    4407 generic.go:356] "Generic (PLEG): container finished" podID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerID="d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e" exitCode=0
Nov 02 03:27:59.014925 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.014830    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"
Nov 02 03:27:59.014925 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.014863    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" event={"ID":"e6e61c64-563d-4c06-b771-89b52ff8191b","Type":"ContainerDied","Data":"d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e"}
Nov 02 03:27:59.015046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.014927    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85" event={"ID":"e6e61c64-563d-4c06-b771-89b52ff8191b","Type":"ContainerDied","Data":"04885c1ebd0f4c2fd74694b156918ac41b28c815fb735d7fe20ae67505ddf17c"}
Nov 02 03:27:59.015046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.014957    4407 scope.go:122] "RemoveContainer" containerID="d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e"
Nov 02 03:27:59.029148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.027118    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:59.029148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.027153    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:59.029148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.027167    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:59.029148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.027180    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6fxw\" (UniqueName: \"kubernetes.io/projected/e6e61c64-563d-4c06-b771-89b52ff8191b-kube-api-access-j6fxw\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:59.029148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.027199    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e61c64-563d-4c06-b771-89b52ff8191b-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:59.029148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.027226    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e61c64-563d-4c06-b771-89b52ff8191b-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:27:59.029148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.029013    4407 scope.go:122] "RemoveContainer" containerID="15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a"
Nov 02 03:27:59.038414 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.038394    4407 scope.go:122] "RemoveContainer" containerID="d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e"
Nov 02 03:27:59.038708 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:59.038685    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e\": container with ID starting with d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e not found: ID does not exist" containerID="d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e"
Nov 02 03:27:59.038775 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.038718    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e"} err="failed to get container status \"d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e\": rpc error: code = NotFound desc = could not find container \"d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e\": container with ID starting with d6ad823443e276acf1af8ecfccdbea3e56eb66a956723c32657eb22b8bf8b18e not found: ID does not exist"
Nov 02 03:27:59.038775 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.038743    4407 scope.go:122] "RemoveContainer" containerID="15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a"
Nov 02 03:27:59.039023 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:27:59.039004    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a\": container with ID starting with 15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a not found: ID does not exist" containerID="15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a"
Nov 02 03:27:59.039071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.039027    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a"} err="failed to get container status \"15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a\": rpc error: code = NotFound desc = could not find container \"15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a\": container with ID starting with 15670a37a23de981429c03f63dbe7fbf3ec88d54e8ee39597cbfc3f2df6b8d7a not found: ID does not exist"
Nov 02 03:27:59.131466 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.131417    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podStartSLOduration=2.131404349 podStartE2EDuration="2.131404349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:27:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:27:59.129927957 +0000 UTC m=+2422.925276522" watchObservedRunningTime="2026-11-02 03:27:59.131404349 +0000 UTC m=+2422.926752936"
Nov 02 03:27:59.257982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.257947    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"]
Nov 02 03:27:59.262183 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.262155    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a27da5-openshift-default-68bb95fc4c-rbh85"]
Nov 02 03:27:59.515465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.515435    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:27:59.517025 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.517000    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-q9rnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 03:27:59.517131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.517047    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 03:27:59.925273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:27:59.925185    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:00.113276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:00.113244    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:00.276821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:00.276778    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:00.350980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:00.350948    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" path="/var/lib/kubelet/pods/e6e61c64-563d-4c06-b771-89b52ff8191b/volumes"
Nov 02 03:28:00.515696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:00.515666    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-q9rnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 03:28:00.515866 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:00.515717    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 03:28:01.515180 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:01.515142    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-q9rnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 03:28:01.515613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:01.515196    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 03:28:02.515190 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:02.515152    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-q9rnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 03:28:02.515572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:02.515238    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 03:28:03.001247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.001177    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:03.227539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.227506    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:03.408348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.408247    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:03.515344 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.515312    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-q9rnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 03:28:03.515971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.515938    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 03:28:03.622075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.622033    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"]
Nov 02 03:28:03.622816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.622796    4407 cpu_manager.go:414] "containerMap: removing container" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy"
Nov 02 03:28:03.622976 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.622963    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy"
Nov 02 03:28:03.623072 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.623062    4407 cpu_manager.go:414] "containerMap: removing container" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy"
Nov 02 03:28:03.623141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.623131    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy"
Nov 02 03:28:03.623422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.623411    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy"
Nov 02 03:28:03.623503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.623494    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6e61c64-563d-4c06-b771-89b52ff8191b" containerName="istio-proxy"
Nov 02 03:28:03.627408 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.627389    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:03.679752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.679669    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"]
Nov 02 03:28:03.714113 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.714076    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74\" (UID: \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:03.714264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.714122    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsx82\" (UniqueName: \"kubernetes.io/projected/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-kube-api-access-jsx82\") pod \"payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74\" (UID: \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:03.727040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.726981    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-dd87f0\""
Nov 02 03:28:03.727153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.727034    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-dd87f0-dockercfg-vpn24\""
Nov 02 03:28:03.772645 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.772116    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"]
Nov 02 03:28:03.778030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.777062    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:03.849666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.846871    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"]
Nov 02 03:28:03.857232 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.857202    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xwvn\" (UniqueName: \"kubernetes.io/projected/15b3a067-d14c-4f00-9f35-d872a57bdd46-kube-api-access-5xwvn\") pod \"payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg\" (UID: \"15b3a067-d14c-4f00-9f35-d872a57bdd46\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:03.857417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.857306    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74\" (UID: \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:03.857417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.857339    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsx82\" (UniqueName: \"kubernetes.io/projected/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-kube-api-access-jsx82\") pod \"payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74\" (UID: \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:03.857417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.857384    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15b3a067-d14c-4f00-9f35-d872a57bdd46-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg\" (UID: \"15b3a067-d14c-4f00-9f35-d872a57bdd46\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:03.858459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.858182    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74\" (UID: \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:03.919662 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.917208    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsx82\" (UniqueName: \"kubernetes.io/projected/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-kube-api-access-jsx82\") pod \"payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74\" (UID: \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:03.949583 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.949510    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:03.959066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.959035    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15b3a067-d14c-4f00-9f35-d872a57bdd46-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg\" (UID: \"15b3a067-d14c-4f00-9f35-d872a57bdd46\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:03.959224 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.959211    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwvn\" (UniqueName: \"kubernetes.io/projected/15b3a067-d14c-4f00-9f35-d872a57bdd46-kube-api-access-5xwvn\") pod \"payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg\" (UID: \"15b3a067-d14c-4f00-9f35-d872a57bdd46\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:03.959768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:03.959747    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15b3a067-d14c-4f00-9f35-d872a57bdd46-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg\" (UID: \"15b3a067-d14c-4f00-9f35-d872a57bdd46\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:04.021663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:04.018830    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xwvn\" (UniqueName: \"kubernetes.io/projected/15b3a067-d14c-4f00-9f35-d872a57bdd46-kube-api-access-5xwvn\") pod \"payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg\" (UID: \"15b3a067-d14c-4f00-9f35-d872a57bdd46\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:04.137040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:04.137005    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:04.335648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:04.333340    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"]
Nov 02 03:28:04.339760 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:04.339732    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20eb9978_ed21_4b9a_9c54_5652c26aa8d1.slice/crio-741749403df182d17a4bd63c838def04f470f9b7f852034aa93127050b7a115a WatchSource:0}: Error finding container 741749403df182d17a4bd63c838def04f470f9b7f852034aa93127050b7a115a: Status 404 returned error can't find the container with id 741749403df182d17a4bd63c838def04f470f9b7f852034aa93127050b7a115a
Nov 02 03:28:04.341532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:04.341512    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 03:28:04.515419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:04.515390    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-q9rnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 03:28:04.515740 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:04.515467    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 03:28:04.515820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:04.515800    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"]
Nov 02 03:28:04.517093 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:04.517055    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b3a067_d14c_4f00_9f35_d872a57bdd46.slice/crio-35f15b6df79753e80a30602f38db9a4c59f45300e1e5051c5ac344de544ab1df WatchSource:0}: Error finding container 35f15b6df79753e80a30602f38db9a4c59f45300e1e5051c5ac344de544ab1df: Status 404 returned error can't find the container with id 35f15b6df79753e80a30602f38db9a4c59f45300e1e5051c5ac344de544ab1df
Nov 02 03:28:05.057186 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:05.057148    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg" event={"ID":"15b3a067-d14c-4f00-9f35-d872a57bdd46","Type":"ContainerStarted","Data":"35f15b6df79753e80a30602f38db9a4c59f45300e1e5051c5ac344de544ab1df"}
Nov 02 03:28:05.058459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:05.058433    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74" event={"ID":"20eb9978-ed21-4b9a-9c54-5652c26aa8d1","Type":"ContainerStarted","Data":"741749403df182d17a4bd63c838def04f470f9b7f852034aa93127050b7a115a"}
Nov 02 03:28:05.188984 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:28:05.188941    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:28:05.515529 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:05.515491    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-q9rnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 03:28:05.516038 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:05.515565    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 03:28:06.621492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:06.621456    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:28:06.622476 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:06.622452    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:28:06.623373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:06.623336    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:28:07.950852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:07.950762    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:08.081445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:08.081408    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74" event={"ID":"20eb9978-ed21-4b9a-9c54-5652c26aa8d1","Type":"ContainerStarted","Data":"841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543"}
Nov 02 03:28:08.081686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:08.081549    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:08.197922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:08.197871    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74" podStartSLOduration=1.850326841 podStartE2EDuration="5.197858381s" totalImagesPullingTime="3.34753154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:08.196145543 +0000 UTC m=+2431.991494110" watchObservedRunningTime="2026-11-02 03:28:08.197858381 +0000 UTC m=+2431.993206945"
Nov 02 03:28:09.089340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:09.089302    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg" event={"ID":"15b3a067-d14c-4f00-9f35-d872a57bdd46","Type":"ContainerStarted","Data":"f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4"}
Nov 02 03:28:09.211673 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:09.211588    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg" podStartSLOduration=2.038603699 podStartE2EDuration="6.211567335s" totalImagesPullingTime="4.172963636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:09.209096069 +0000 UTC m=+2433.004444635" watchObservedRunningTime="2026-11-02 03:28:09.211567335 +0000 UTC m=+2433.006915901"
Nov 02 03:28:10.096295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:10.096268    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:13.240108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:13.240076    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:17.465574 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:17.465538    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:17.906774 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:17.906743    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"]
Nov 02 03:28:17.912851 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:17.912819    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:17.969187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:17.969147    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-045653-openshift-default-dockercfg-strg5\""
Nov 02 03:28:18.040965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.040914    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:18.041849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.041816    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-podinfo\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.042001 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.041976    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-envoy\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.042261 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.042233    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-credential-socket\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.042354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.042275    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8ea9264-44db-4610-ba04-95a8ba22a66a-istiod-ca-cert\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.042354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.042304    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q29w\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-kube-api-access-8q29w\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.042450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.042406    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-certs\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.042499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.042446    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-socket\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.042499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.042472    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-token\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.042592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.042547    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-data\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.045500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.045453    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"]
Nov 02 03:28:18.145308 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145276    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-podinfo\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.145532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145369    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-envoy\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.145532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145427    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-credential-socket\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.145532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145450    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8ea9264-44db-4610-ba04-95a8ba22a66a-istiod-ca-cert\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.145532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145475    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q29w\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-kube-api-access-8q29w\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.145532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145516    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-certs\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.145876 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145543    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-socket\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.145876 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145567    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-token\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.145876 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.145761    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-data\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.148063 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.148040    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-data\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.148485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.148467    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8ea9264-44db-4610-ba04-95a8ba22a66a-istiod-ca-cert\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.148745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.148722    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-certs\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.149033 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.149013    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-credential-socket\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.149225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.149198    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-socket\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.153545 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.153512    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-envoy\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.156829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.156775    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-podinfo\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.209577 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.209551    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q29w\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-kube-api-access-8q29w\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.213406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.213381    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-token\") pod \"e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") " pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.228588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.228557    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:18.346652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.346592    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:18.503292 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.503262    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:18.661001 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.660966    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"]
Nov 02 03:28:18.701175 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:18.701148    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ea9264_44db_4610_ba04_95a8ba22a66a.slice/crio-4adc8c9b79abe55af1a1df0e7e47b749747322d005b3f252445479528f261376 WatchSource:0}: Error finding container 4adc8c9b79abe55af1a1df0e7e47b749747322d005b3f252445479528f261376: Status 404 returned error can't find the container with id 4adc8c9b79abe55af1a1df0e7e47b749747322d005b3f252445479528f261376
Nov 02 03:28:18.703611 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.703578    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:28:18.703725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.703670    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:28:18.703764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:18.703721    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:28:19.093114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:19.093037    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:28:19.156509 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:19.156478    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" event={"ID":"a8ea9264-44db-4610-ba04-95a8ba22a66a","Type":"ContainerStarted","Data":"9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a"}
Nov 02 03:28:19.156697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:19.156528    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" event={"ID":"a8ea9264-44db-4610-ba04-95a8ba22a66a","Type":"ContainerStarted","Data":"4adc8c9b79abe55af1a1df0e7e47b749747322d005b3f252445479528f261376"}
Nov 02 03:28:19.229105 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:19.229075    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:19.236077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:19.236051    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:19.329067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:19.329017    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" podStartSLOduration=2.329001351 podStartE2EDuration="2.329001351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:19.326946498 +0000 UTC m=+2443.122295064" watchObservedRunningTime="2026-11-02 03:28:19.329001351 +0000 UTC m=+2443.124349916"
Nov 02 03:28:20.162773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:20.162743    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:20.163814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:20.163795    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:28:21.106029 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:21.106002    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:28:22.083802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:22.083747    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:22.261962 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:22.261932    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:22.426995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:22.426918    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:23.307433 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:23.307399    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:25.311566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.311540    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:25.514481 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.514444    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:25.870395 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.870354    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"]
Nov 02 03:28:25.886941 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.886913    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:25.918277 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.918249    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"]
Nov 02 03:28:25.951301 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.951269    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p664b\" (UniqueName: \"kubernetes.io/projected/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-kube-api-access-p664b\") pod \"payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f\" (UID: \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:25.951604 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.951580    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f\" (UID: \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:25.965908 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.965882    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-045653\""
Nov 02 03:28:25.966062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:25.965905    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-045653-dockercfg-9mm8h\""
Nov 02 03:28:26.017295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.017258    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"]
Nov 02 03:28:26.021282 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.021266    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:26.052633 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.052601    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f\" (UID: \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:26.052798 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.052666    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwj5d\" (UniqueName: \"kubernetes.io/projected/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-kube-api-access-xwj5d\") pod \"payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5\" (UID: \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:26.052798 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.052761    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p664b\" (UniqueName: \"kubernetes.io/projected/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-kube-api-access-p664b\") pod \"payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f\" (UID: \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:26.052884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.052866    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5\" (UID: \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:26.053360 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.053342    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f\" (UID: \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:26.078818 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.078793    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"]
Nov 02 03:28:26.111140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.111104    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p664b\" (UniqueName: \"kubernetes.io/projected/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-kube-api-access-p664b\") pod \"payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f\" (UID: \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:26.154191 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.154108    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwj5d\" (UniqueName: \"kubernetes.io/projected/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-kube-api-access-xwj5d\") pod \"payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5\" (UID: \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:26.154342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.154266    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5\" (UID: \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:26.154986 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.154966    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5\" (UID: \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:26.206782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.206751    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:26.212483 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.212446    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwj5d\" (UniqueName: \"kubernetes.io/projected/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-kube-api-access-xwj5d\") pod \"payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5\" (UID: \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:26.334917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.334885    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:26.598270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.598243    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"]
Nov 02 03:28:26.599001 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:26.598977    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3292ca90_b8ac_4ef1_98f4_4d180d777f4c.slice/crio-fec984d0ffa92ed37da34d6d32e8dcca4e90c3ff773d73df06334ff3b7692644 WatchSource:0}: Error finding container fec984d0ffa92ed37da34d6d32e8dcca4e90c3ff773d73df06334ff3b7692644: Status 404 returned error can't find the container with id fec984d0ffa92ed37da34d6d32e8dcca4e90c3ff773d73df06334ff3b7692644
Nov 02 03:28:26.718138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:26.718111    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"]
Nov 02 03:28:26.718906 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:26.718879    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee2ad395_ff5b_4b7e_bbf2_eac5e68648f9.slice/crio-11be9dee2a3a9635aac9d0fa8b83ea62095b94fd367b6aedb4bb1e43988026a4 WatchSource:0}: Error finding container 11be9dee2a3a9635aac9d0fa8b83ea62095b94fd367b6aedb4bb1e43988026a4: Status 404 returned error can't find the container with id 11be9dee2a3a9635aac9d0fa8b83ea62095b94fd367b6aedb4bb1e43988026a4
Nov 02 03:28:27.206146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:27.206109    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f" event={"ID":"3292ca90-b8ac-4ef1-98f4-4d180d777f4c","Type":"ContainerStarted","Data":"c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774"}
Nov 02 03:28:27.206342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:27.206205    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:27.206342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:27.206226    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f" event={"ID":"3292ca90-b8ac-4ef1-98f4-4d180d777f4c","Type":"ContainerStarted","Data":"fec984d0ffa92ed37da34d6d32e8dcca4e90c3ff773d73df06334ff3b7692644"}
Nov 02 03:28:27.208036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:27.208013    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5" event={"ID":"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9","Type":"ContainerStarted","Data":"3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d"}
Nov 02 03:28:27.208128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:27.208053    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5" event={"ID":"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9","Type":"ContainerStarted","Data":"11be9dee2a3a9635aac9d0fa8b83ea62095b94fd367b6aedb4bb1e43988026a4"}
Nov 02 03:28:27.208188 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:27.208169    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:27.321491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:27.321431    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f" podStartSLOduration=2.321416559 podStartE2EDuration="2.321416559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:27.319797175 +0000 UTC m=+2451.115145741" watchObservedRunningTime="2026-11-02 03:28:27.321416559 +0000 UTC m=+2451.116765124"
Nov 02 03:28:27.435918 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:27.435861    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5" podStartSLOduration=2.43584031 podStartE2EDuration="2.43584031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:27.433282821 +0000 UTC m=+2451.228631385" watchObservedRunningTime="2026-11-02 03:28:27.43584031 +0000 UTC m=+2451.231188876"
Nov 02 03:28:28.363702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.363665    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"]
Nov 02 03:28:28.367798 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.367775    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.420693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.420669    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-4c3ea5-serving-cert\""
Nov 02 03:28:28.475663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.474974    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"]
Nov 02 03:28:28.482376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.482351    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.482530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.482393    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.482530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.482512    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcw8q\" (UniqueName: \"kubernetes.io/projected/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-kube-api-access-fcw8q\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.482669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.482560    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.580843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.580815    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:28.583387 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.583363    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.583490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.583407    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.583539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.583527    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcw8q\" (UniqueName: \"kubernetes.io/projected/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-kube-api-access-fcw8q\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.583601 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:28:28.583579    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-serving-cert: secret "maas-api-e2e-shared-a-w2-4c3ea5-serving-cert" not found
Nov 02 03:28:28.583667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.583605    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.583667 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:28:28.583663    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls podName:4910eacd-0342-4cd2-ad5d-fc5ead694dbb nodeName:}" failed. No retries permitted until 2026-11-02 03:28:29.08364422 +0000 UTC m=+2452.878992764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls") pod "maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts" (UID: "4910eacd-0342-4cd2-ad5d-fc5ead694dbb") : secret "maas-api-e2e-shared-a-w2-4c3ea5-serving-cert" not found
Nov 02 03:28:28.584255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.584213    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.585898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.585881    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.645223 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.645096    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcw8q\" (UniqueName: \"kubernetes.io/projected/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-kube-api-access-fcw8q\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:28.773123 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.773085    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:28.951212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:28.951132    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:29.089369 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:29.089325    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:29.092114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:29.092084    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:29.281290 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:29.281263    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:29.656785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:29.656760    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"]
Nov 02 03:28:29.657589 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:29.657563    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4910eacd_0342_4cd2_ad5d_fc5ead694dbb.slice/crio-f4c68dbbfbd426bc38a3d78886c6b25f44e1aa0128cca470940c2287157da402 WatchSource:0}: Error finding container f4c68dbbfbd426bc38a3d78886c6b25f44e1aa0128cca470940c2287157da402: Status 404 returned error can't find the container with id f4c68dbbfbd426bc38a3d78886c6b25f44e1aa0128cca470940c2287157da402
Nov 02 03:28:30.236712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:30.236676    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts" event={"ID":"4910eacd-0342-4cd2-ad5d-fc5ead694dbb","Type":"ContainerStarted","Data":"f4c68dbbfbd426bc38a3d78886c6b25f44e1aa0128cca470940c2287157da402"}
Nov 02 03:28:30.784266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:30.784191    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:28:31.244813 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:31.244782    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts" event={"ID":"4910eacd-0342-4cd2-ad5d-fc5ead694dbb","Type":"ContainerStarted","Data":"338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e"}
Nov 02 03:28:31.245007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:31.244879    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:31.372090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:31.372027    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts" podStartSLOduration=2.525331087 podStartE2EDuration="3.372007043s" totalImagesPullingTime="846.675956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:31.369184812 +0000 UTC m=+2455.164533397" watchObservedRunningTime="2026-11-02 03:28:31.372007043 +0000 UTC m=+2455.167355609"
Nov 02 03:28:33.526881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:33.526824    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:35.163185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:35.163149    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:35.190053 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:28:35.190003    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:28:35.328965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:35.328936    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:37.257096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:37.257062    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 03:28:37.708153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:37.708075    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:38.217313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:38.217282    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:28:38.217551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:38.217375    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:28:39.284221 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:39.284191    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:39.583536 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:39.583460    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:39.739243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:39.739209    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:40.250491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:40.250463    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:42.112904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:42.112865    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:42.356106 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:42.356069    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:42.853141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:42.853108    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:43.464738 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:43.464604    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:43.956855 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:43.956825    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:44.532642 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.532591    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"]
Nov 02 03:28:44.540503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.540481    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.596558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.596520    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"]
Nov 02 03:28:44.640767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.640740    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-dd87f0-serving-cert\""
Nov 02 03:28:44.709028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.708997    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qrk\" (UniqueName: \"kubernetes.io/projected/924714d8-000e-4584-b58e-d9631ac77a20-kube-api-access-j6qrk\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.709184 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.709054    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/924714d8-000e-4584-b58e-d9631ac77a20-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.709184 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.709122    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-metrics-certs\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.709281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.709266    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.751580 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.751543    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:44.810194 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.810103    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qrk\" (UniqueName: \"kubernetes.io/projected/924714d8-000e-4584-b58e-d9631ac77a20-kube-api-access-j6qrk\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.810194 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.810183    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/924714d8-000e-4584-b58e-d9631ac77a20-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.810429 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.810210    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-metrics-certs\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.810429 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.810358    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.810512 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:28:44.810475    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-serving-cert: secret "maas-api-e2e-ipp-a-dd87f0-serving-cert" not found
Nov 02 03:28:44.810561 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:28:44.810548    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls podName:924714d8-000e-4584-b58e-d9631ac77a20 nodeName:}" failed. No retries permitted until 2026-11-02 03:28:45.310531442 +0000 UTC m=+2469.105879999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls") pod "maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6" (UID: "924714d8-000e-4584-b58e-d9631ac77a20") : secret "maas-api-e2e-ipp-a-dd87f0-serving-cert" not found
Nov 02 03:28:44.810934 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.810913    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/924714d8-000e-4584-b58e-d9631ac77a20-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.813089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.813069    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-metrics-certs\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.874268 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.874239    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qrk\" (UniqueName: \"kubernetes.io/projected/924714d8-000e-4584-b58e-d9631ac77a20-kube-api-access-j6qrk\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:44.912606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:44.912573    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:45.065477 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:45.065384    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:45.317669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:45.317533    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:45.320313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:45.320285    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:45.453810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:45.453781    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:45.835863 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:45.835835    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"]
Nov 02 03:28:45.836406 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:45.836383    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924714d8_000e_4584_b58e_d9631ac77a20.slice/crio-592eba2b22d43afbca1daadfac8d4af0367c79ae5f0085faa35eb5e622feec68 WatchSource:0}: Error finding container 592eba2b22d43afbca1daadfac8d4af0367c79ae5f0085faa35eb5e622feec68: Status 404 returned error can't find the container with id 592eba2b22d43afbca1daadfac8d4af0367c79ae5f0085faa35eb5e622feec68
Nov 02 03:28:46.084326 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:46.084244    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:46.246004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:46.245969    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:46.336397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:46.336309    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6" event={"ID":"924714d8-000e-4584-b58e-d9631ac77a20","Type":"ContainerStarted","Data":"592eba2b22d43afbca1daadfac8d4af0367c79ae5f0085faa35eb5e622feec68"}
Nov 02 03:28:46.369941 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:46.369907    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"]
Nov 02 03:28:46.370134 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:46.370088    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy" containerID="cri-o://fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f" gracePeriod=30
Nov 02 03:28:46.409736 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:46.409702    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:47.343747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:47.343711    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6" event={"ID":"924714d8-000e-4584-b58e-d9631ac77a20","Type":"ContainerStarted","Data":"af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671"}
Nov 02 03:28:47.344144 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:47.343977    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:47.462385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:47.462327    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6" podStartSLOduration=2.597301245 podStartE2EDuration="3.462308745s" totalImagesPullingTime="865.0075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:47.461605955 +0000 UTC m=+2471.256954517" watchObservedRunningTime="2026-11-02 03:28:47.462308745 +0000 UTC m=+2471.257657313"
Nov 02 03:28:47.565820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:47.565787    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:47.721937 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:47.721861    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:47.882348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:47.882307    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:48.041632 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:48.041601    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:48.206040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:48.204978    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:48.530984 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:48.530950    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:50.151829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:50.151795    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:51.634248 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.634184    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:28:51.710450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710414    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-socket\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.710608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710456    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-credential-socket\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.710608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710494    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14d5530f-010d-4fc8-b40f-79c2eb27d763-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d5530f-010d-4fc8-b40f-79c2eb27d763-istiod-ca-cert\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.710608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710518    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-podinfo\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.710608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710551    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-envoy\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.710894 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710657    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-token\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-token\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.710894 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710685    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-data\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.710894 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710727    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-socket" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:28:51.710894 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710769    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-kube-api-access-b4hk6\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-kube-api-access-b4hk6\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.710894 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710838    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-certs\") pod \"14d5530f-010d-4fc8-b40f-79c2eb27d763\" (UID: \"14d5530f-010d-4fc8-b40f-79c2eb27d763\") "
Nov 02 03:28:51.711157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710763    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-credential-socket" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:28:51.711157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710959    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-data" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:28:51.711157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.710949    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14d5530f-010d-4fc8-b40f-79c2eb27d763-istiod-ca-cert" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:28:51.711474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.711257    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-certs" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:28:51.711558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.711516    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:51.711651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.711636    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:51.711745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.711681    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:51.711807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.711750    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d5530f-010d-4fc8-b40f-79c2eb27d763-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:51.711807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.711764    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:51.712861 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.712837    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-envoy" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:28:51.712957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.712941    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-podinfo" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:28:51.713003 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.712972    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-token" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:28:51.713123 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.713109    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-kube-api-access-b4hk6" pod "14d5530f-010d-4fc8-b40f-79c2eb27d763" (UID: "14d5530f-010d-4fc8-b40f-79c2eb27d763"). InnerVolumeSpecName "kube-api-access-b4hk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:28:51.813023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.812990    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:51.813023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.813025    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:51.813207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.813038    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:51.813207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:51.813052    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4hk6\" (UniqueName: \"kubernetes.io/projected/14d5530f-010d-4fc8-b40f-79c2eb27d763-kube-api-access-b4hk6\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:28:52.374400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.374373    4407 generic.go:356] "Generic (PLEG): container finished" podID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerID="fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f" exitCode=0
Nov 02 03:28:52.374539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.374421    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"
Nov 02 03:28:52.374539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.374459    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" event={"ID":"14d5530f-010d-4fc8-b40f-79c2eb27d763","Type":"ContainerDied","Data":"fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f"}
Nov 02 03:28:52.374539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.374517    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp" event={"ID":"14d5530f-010d-4fc8-b40f-79c2eb27d763","Type":"ContainerDied","Data":"3450eb1222671d7a741f5e1cf5c6b0b9b06c31f8801aa60c1e9f963bcf8dfdc0"}
Nov 02 03:28:52.374539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.374538    4407 scope.go:122] "RemoveContainer" containerID="fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f"
Nov 02 03:28:52.384011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.383995    4407 scope.go:122] "RemoveContainer" containerID="392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900"
Nov 02 03:28:52.392954 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.392939    4407 scope.go:122] "RemoveContainer" containerID="fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f"
Nov 02 03:28:52.393240 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:28:52.393221    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f\": container with ID starting with fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f not found: ID does not exist" containerID="fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f"
Nov 02 03:28:52.393288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.393253    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f"} err="failed to get container status \"fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f\": rpc error: code = NotFound desc = could not find container \"fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f\": container with ID starting with fdb31ac122dd0bf3ef4114e11acceb4db900ca2cabdfd3d7e9ac164b955e620f not found: ID does not exist"
Nov 02 03:28:52.393288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.393272    4407 scope.go:122] "RemoveContainer" containerID="392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900"
Nov 02 03:28:52.393518 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:28:52.393500    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900\": container with ID starting with 392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900 not found: ID does not exist" containerID="392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900"
Nov 02 03:28:52.393580 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.393531    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900"} err="failed to get container status \"392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900\": rpc error: code = NotFound desc = could not find container \"392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900\": container with ID starting with 392f1af764afbf98aeb077db33a2b6c9e015f361347727bbf4e68d90aeb84900 not found: ID does not exist"
Nov 02 03:28:52.593455 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.593420    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"]
Nov 02 03:28:52.643787 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:52.643711    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-c170e9-openshift-default-7544f57bdf-wmxwp"]
Nov 02 03:28:53.088065 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.088036    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:53.256702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.256669    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:53.356443 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.356369    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:28:53.474608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.474571    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"]
Nov 02 03:28:53.475247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.475227    4407 cpu_manager.go:414] "containerMap: removing container" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy"
Nov 02 03:28:53.475247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.475247    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy"
Nov 02 03:28:53.475431 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.475416    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy"
Nov 02 03:28:53.475490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.475439    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy"
Nov 02 03:28:53.480262 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.480242    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:53.535727 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.534930    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"]
Nov 02 03:28:53.535890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.535738    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f1a692\""
Nov 02 03:28:53.536728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.536694    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5qm2\" (UniqueName: \"kubernetes.io/projected/e4ead691-2863-43f4-b7d5-b723183a54bf-kube-api-access-x5qm2\") pod \"payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh\" (UID: \"e4ead691-2863-43f4-b7d5-b723183a54bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:53.536899 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.536879    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ead691-2863-43f4-b7d5-b723183a54bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh\" (UID: \"e4ead691-2863-43f4-b7d5-b723183a54bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:53.537866 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.537840    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f1a692-dockercfg-2qtpp\""
Nov 02 03:28:53.655738 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.655650    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5qm2\" (UniqueName: \"kubernetes.io/projected/e4ead691-2863-43f4-b7d5-b723183a54bf-kube-api-access-x5qm2\") pod \"payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh\" (UID: \"e4ead691-2863-43f4-b7d5-b723183a54bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:53.656501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.656473    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ead691-2863-43f4-b7d5-b723183a54bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh\" (UID: \"e4ead691-2863-43f4-b7d5-b723183a54bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:53.657575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.657550    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ead691-2863-43f4-b7d5-b723183a54bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh\" (UID: \"e4ead691-2863-43f4-b7d5-b723183a54bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:53.661157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.661129    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:53.661594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.661575    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"]
Nov 02 03:28:53.662612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.662579    4407 cpu_manager.go:414] "containerMap: removing container" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy"
Nov 02 03:28:53.662612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.662606    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" containerName="istio-proxy"
Nov 02 03:28:53.667938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.667917    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:53.694614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.694576    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"]
Nov 02 03:28:53.715971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.715945    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5qm2\" (UniqueName: \"kubernetes.io/projected/e4ead691-2863-43f4-b7d5-b723183a54bf-kube-api-access-x5qm2\") pod \"payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh\" (UID: \"e4ead691-2863-43f4-b7d5-b723183a54bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:53.757373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.757345    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk8vt\" (UniqueName: \"kubernetes.io/projected/0df298c8-9673-4903-8be0-d11cf1380dd1-kube-api-access-xk8vt\") pod \"payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px\" (UID: \"0df298c8-9673-4903-8be0-d11cf1380dd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:53.757743 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.757566    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0df298c8-9673-4903-8be0-d11cf1380dd1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px\" (UID: \"0df298c8-9673-4903-8be0-d11cf1380dd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:53.799082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.799057    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:53.860544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.860506    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0df298c8-9673-4903-8be0-d11cf1380dd1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px\" (UID: \"0df298c8-9673-4903-8be0-d11cf1380dd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:53.860717 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.860640    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk8vt\" (UniqueName: \"kubernetes.io/projected/0df298c8-9673-4903-8be0-d11cf1380dd1-kube-api-access-xk8vt\") pod \"payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px\" (UID: \"0df298c8-9673-4903-8be0-d11cf1380dd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:53.862033 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.862002    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0df298c8-9673-4903-8be0-d11cf1380dd1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px\" (UID: \"0df298c8-9673-4903-8be0-d11cf1380dd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:53.877671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.877575    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:53.918800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.918727    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk8vt\" (UniqueName: \"kubernetes.io/projected/0df298c8-9673-4903-8be0-d11cf1380dd1-kube-api-access-xk8vt\") pod \"payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px\" (UID: \"0df298c8-9673-4903-8be0-d11cf1380dd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:53.986757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:53.986725    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:54.048584 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.048561    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:54.181082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.181050    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"]
Nov 02 03:28:54.181977 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:54.181950    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ead691_2863_43f4_b7d5_b723183a54bf.slice/crio-8236bb60659ffc24a987507ca63816ecec19225a2cc2240d65c2762bf8e365c4 WatchSource:0}: Error finding container 8236bb60659ffc24a987507ca63816ecec19225a2cc2240d65c2762bf8e365c4: Status 404 returned error can't find the container with id 8236bb60659ffc24a987507ca63816ecec19225a2cc2240d65c2762bf8e365c4
Nov 02 03:28:54.351721 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.351692    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14d5530f-010d-4fc8-b40f-79c2eb27d763" path="/var/lib/kubelet/pods/14d5530f-010d-4fc8-b40f-79c2eb27d763/volumes"
Nov 02 03:28:54.360304 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.360284    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"]
Nov 02 03:28:54.361106 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:54.361071    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df298c8_9673_4903_8be0_d11cf1380dd1.slice/crio-8a5428d98843b0fd179a681e53a155122b5ae4390926fbb5ebb9d07ee208ffbc WatchSource:0}: Error finding container 8a5428d98843b0fd179a681e53a155122b5ae4390926fbb5ebb9d07ee208ffbc: Status 404 returned error can't find the container with id 8a5428d98843b0fd179a681e53a155122b5ae4390926fbb5ebb9d07ee208ffbc
Nov 02 03:28:54.388184 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.388151    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh" event={"ID":"e4ead691-2863-43f4-b7d5-b723183a54bf","Type":"ContainerStarted","Data":"4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c"}
Nov 02 03:28:54.388309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.388247    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh" event={"ID":"e4ead691-2863-43f4-b7d5-b723183a54bf","Type":"ContainerStarted","Data":"8236bb60659ffc24a987507ca63816ecec19225a2cc2240d65c2762bf8e365c4"}
Nov 02 03:28:54.388418 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.388392    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:28:54.389439 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.389414    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px" event={"ID":"0df298c8-9673-4903-8be0-d11cf1380dd1","Type":"ContainerStarted","Data":"8a5428d98843b0fd179a681e53a155122b5ae4390926fbb5ebb9d07ee208ffbc"}
Nov 02 03:28:54.504966 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:54.504909    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh" podStartSLOduration=1.504894508 podStartE2EDuration="1.504894508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:54.503834821 +0000 UTC m=+2478.299183386" watchObservedRunningTime="2026-11-02 03:28:54.504894508 +0000 UTC m=+2478.300243073"
Nov 02 03:28:55.396535 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:55.396501    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px" event={"ID":"0df298c8-9673-4903-8be0-d11cf1380dd1","Type":"ContainerStarted","Data":"cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc"}
Nov 02 03:28:55.397065 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:55.396648    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:28:55.439050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:55.439021    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:55.561585 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:55.561534    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px" podStartSLOduration=2.561518646 podStartE2EDuration="2.561518646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:55.510495118 +0000 UTC m=+2479.305843684" watchObservedRunningTime="2026-11-02 03:28:55.561518646 +0000 UTC m=+2479.356867236"
Nov 02 03:28:57.959062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:57.959029    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:58.121820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.121786    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:28:58.285140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.285106    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"]
Nov 02 03:28:58.289475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.289454    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:58.342675 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.342614    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-abc348\""
Nov 02 03:28:58.342864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.342846    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-abc348-dockercfg-5r5f5\""
Nov 02 03:28:58.396649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.372448    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"]
Nov 02 03:28:58.427868 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.427832    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dce5475-f780-4807-b644-0914f37810a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl\" (UID: \"0dce5475-f780-4807-b644-0914f37810a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:58.429931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.429897    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8d6p\" (UniqueName: \"kubernetes.io/projected/0dce5475-f780-4807-b644-0914f37810a6-kube-api-access-w8d6p\") pod \"payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl\" (UID: \"0dce5475-f780-4807-b644-0914f37810a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:58.443234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.443192    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"]
Nov 02 03:28:58.448502 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.448479    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:28:58.501474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.501438    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"]
Nov 02 03:28:58.531785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.531747    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8d6p\" (UniqueName: \"kubernetes.io/projected/0dce5475-f780-4807-b644-0914f37810a6-kube-api-access-w8d6p\") pod \"payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl\" (UID: \"0dce5475-f780-4807-b644-0914f37810a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:58.531949 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.531931    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dce5475-f780-4807-b644-0914f37810a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl\" (UID: \"0dce5475-f780-4807-b644-0914f37810a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:58.532720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.532695    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dce5475-f780-4807-b644-0914f37810a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl\" (UID: \"0dce5475-f780-4807-b644-0914f37810a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:58.589390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.589323    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8d6p\" (UniqueName: \"kubernetes.io/projected/0dce5475-f780-4807-b644-0914f37810a6-kube-api-access-w8d6p\") pod \"payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl\" (UID: \"0dce5475-f780-4807-b644-0914f37810a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:58.604269 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.604247    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:58.632910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.632879    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35b2436f-d6f4-427e-994d-4515e582bcde-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq\" (UID: \"35b2436f-d6f4-427e-994d-4515e582bcde\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:28:58.633050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.632943    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbbbw\" (UniqueName: \"kubernetes.io/projected/35b2436f-d6f4-427e-994d-4515e582bcde-kube-api-access-bbbbw\") pod \"payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq\" (UID: \"35b2436f-d6f4-427e-994d-4515e582bcde\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:28:58.734843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.734807    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35b2436f-d6f4-427e-994d-4515e582bcde-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq\" (UID: \"35b2436f-d6f4-427e-994d-4515e582bcde\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:28:58.735019 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.734859    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbbbw\" (UniqueName: \"kubernetes.io/projected/35b2436f-d6f4-427e-994d-4515e582bcde-kube-api-access-bbbbw\") pod \"payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq\" (UID: \"35b2436f-d6f4-427e-994d-4515e582bcde\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:28:58.735492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.735471    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35b2436f-d6f4-427e-994d-4515e582bcde-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq\" (UID: \"35b2436f-d6f4-427e-994d-4515e582bcde\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:28:58.792498 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.792471    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbbbw\" (UniqueName: \"kubernetes.io/projected/35b2436f-d6f4-427e-994d-4515e582bcde-kube-api-access-bbbbw\") pod \"payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq\" (UID: \"35b2436f-d6f4-427e-994d-4515e582bcde\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:28:58.976664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:58.976635    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"]
Nov 02 03:28:58.977332 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:58.977306    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dce5475_f780_4807_b644_0914f37810a6.slice/crio-57fbbd66ac0d5007fb52752974438a195e64c7a9a89e2db2b6abfe5d5ecc593b WatchSource:0}: Error finding container 57fbbd66ac0d5007fb52752974438a195e64c7a9a89e2db2b6abfe5d5ecc593b: Status 404 returned error can't find the container with id 57fbbd66ac0d5007fb52752974438a195e64c7a9a89e2db2b6abfe5d5ecc593b
Nov 02 03:28:59.069017 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:59.068988    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:28:59.445710 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:59.445674    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl" event={"ID":"0dce5475-f780-4807-b644-0914f37810a6","Type":"ContainerStarted","Data":"fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0"}
Nov 02 03:28:59.445917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:59.445735    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl" event={"ID":"0dce5475-f780-4807-b644-0914f37810a6","Type":"ContainerStarted","Data":"57fbbd66ac0d5007fb52752974438a195e64c7a9a89e2db2b6abfe5d5ecc593b"}
Nov 02 03:28:59.445917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:59.445831    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:28:59.454598 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:59.454574    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"]
Nov 02 03:28:59.455481 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:28:59.455462    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b2436f_d6f4_427e_994d_4515e582bcde.slice/crio-73076414e569ce0a1785fb24b758bf9302fb967affdfc7dacabdd30e1fa74d9d WatchSource:0}: Error finding container 73076414e569ce0a1785fb24b758bf9302fb967affdfc7dacabdd30e1fa74d9d: Status 404 returned error can't find the container with id 73076414e569ce0a1785fb24b758bf9302fb967affdfc7dacabdd30e1fa74d9d
Nov 02 03:28:59.563302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:28:59.563244    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl" podStartSLOduration=1.563230938 podStartE2EDuration="1.563230938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:28:59.560112081 +0000 UTC m=+2483.355460649" watchObservedRunningTime="2026-11-02 03:28:59.563230938 +0000 UTC m=+2483.358579503"
Nov 02 03:29:00.052923 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:00.052885    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:00.453661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:00.453548    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq" event={"ID":"35b2436f-d6f4-427e-994d-4515e582bcde","Type":"ContainerStarted","Data":"a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db"}
Nov 02 03:29:00.453840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:00.453825    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:29:00.453951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:00.453866    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq" event={"ID":"35b2436f-d6f4-427e-994d-4515e582bcde","Type":"ContainerStarted","Data":"73076414e569ce0a1785fb24b758bf9302fb967affdfc7dacabdd30e1fa74d9d"}
Nov 02 03:29:00.570294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:00.570244    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq" podStartSLOduration=2.570229881 podStartE2EDuration="2.570229881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:29:00.568114285 +0000 UTC m=+2484.363462849" watchObservedRunningTime="2026-11-02 03:29:00.570229881 +0000 UTC m=+2484.365578445"
Nov 02 03:29:00.871274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:00.871241    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:05.189215 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:29:05.189173    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:29:05.400750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:05.400721    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:29:05.694447 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:05.694421    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:06.405151 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:06.405119    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:29:07.836470 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:07.836440    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:10.348862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:10.348834    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:10.456835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:10.456811    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:29:10.502083 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:10.502046    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:10.658092 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:10.658007    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:11.472702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:11.472671    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:29:11.606390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:11.606353    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:29:15.361519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:15.361488    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:17.326901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:17.326865    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:17.490176 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:17.490085    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:20.184270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:20.184232    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:21.584889 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:21.584846    4407 generic.go:356] "Generic (PLEG): container finished" podID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerID="09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701" exitCode=0
Nov 02 03:29:21.585321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:21.584920    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" event={"ID":"06c937a7-e688-49e8-ab6e-6726e3afa7a9","Type":"ContainerDied","Data":"09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701"}
Nov 02 03:29:21.585321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:21.585315    4407 scope.go:122] "RemoveContainer" containerID="09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701"
Nov 02 03:29:21.587040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:21.587006    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:21.587139 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:21.587095    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:21.587182 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:21.587162    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:22.610665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:22.610604    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" event={"ID":"06c937a7-e688-49e8-ab6e-6726e3afa7a9","Type":"ContainerStarted","Data":"a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22"}
Nov 02 03:29:23.436680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:23.436649    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:29:23.438014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:23.437988    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 03:29:23.438156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:23.438042    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 03:29:24.437534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:24.437500    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 03:29:24.437977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:24.437552    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 03:29:25.061889 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:25.061851    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:25.437440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:25.437363    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 03:29:25.437440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:25.437421    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 03:29:26.437450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:26.437401    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 03:29:26.437678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:26.437469    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 03:29:26.614909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:26.614867    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:27.039462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.039429    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:27.436503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.436422    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 03:29:27.436503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.436482    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 03:29:27.819749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.819689    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"]
Nov 02 03:29:27.838143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.838116    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.862878 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.862853    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-data\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.863110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.863095    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-credential-socket\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.863218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.863205    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-certs\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.863329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.863318    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxgdr\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-kube-api-access-hxgdr\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.863476 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.863457    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istiod-ca-cert\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.863725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.863686    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-socket\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.863969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.863952    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-token\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.864138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.864124    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-envoy\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.864365 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.864351    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-podinfo\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.916651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.895786    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-d94cc3-openshift-default-dockercfg-29dlz\""
Nov 02 03:29:27.939676 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.939505    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:27.952116 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.952018    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"]
Nov 02 03:29:27.965901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.965756    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-token\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.966756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.966727    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-envoy\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.967531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.967507    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-podinfo\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.968171 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.968154    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-data\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.968882 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.968864    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-credential-socket\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.969323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.969303    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-certs\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.969787 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.969768    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxgdr\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-kube-api-access-hxgdr\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.970437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.970416    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istiod-ca-cert\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.973395 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.973377    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-socket\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.974009 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.968730    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-data\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.974155 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.971222    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istiod-ca-cert\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.974270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.969712    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-certs\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.974370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.973280    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-podinfo\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.974462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.969251    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-credential-socket\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:27.974596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:27.973806    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-socket\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:28.034343 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.031743    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:29:28.038195 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.038142    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-envoy\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:28.038994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.038948    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxgdr\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-kube-api-access-hxgdr\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:28.041150 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.041007    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-token\") pod \"e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") " pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:28.152327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.152238    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:28.208651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.206395    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:28.440924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.440845    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:29:28.659175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.659143    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:29:28.708327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.708251    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"]
Nov 02 03:29:28.713210 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:29:28.713185    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f3f2f6_d2ac_41f8_a180_39f32be5bcd3.slice/crio-cef6cf89d70a37b920b09263fcce99118ea3da81b1470991319d0336ff5f611f WatchSource:0}: Error finding container cef6cf89d70a37b920b09263fcce99118ea3da81b1470991319d0336ff5f611f: Status 404 returned error can't find the container with id cef6cf89d70a37b920b09263fcce99118ea3da81b1470991319d0336ff5f611f
Nov 02 03:29:28.715498 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.715470    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:28.715575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.715530    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:28.715575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:28.715559    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:29.665845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:29.665805    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" event={"ID":"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3","Type":"ContainerStarted","Data":"6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd"}
Nov 02 03:29:29.666331 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:29.665870    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" event={"ID":"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3","Type":"ContainerStarted","Data":"cef6cf89d70a37b920b09263fcce99118ea3da81b1470991319d0336ff5f611f"}
Nov 02 03:29:29.800649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:29.788547    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" podStartSLOduration=2.788512806 podStartE2EDuration="2.788512806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:29:29.786329251 +0000 UTC m=+2513.581677817" watchObservedRunningTime="2026-11-02 03:29:29.788512806 +0000 UTC m=+2513.583861372"
Nov 02 03:29:30.152501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:30.152467    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:30.158339 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:30.158315    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:30.372046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:30.372008    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:30.672849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:30.672823    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:30.673945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:30.673927    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:29:32.600937 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:32.600896    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:32.812176 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:32.812140    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:33.017744 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:33.017712    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:35.190617 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:29:35.190575    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:29:35.651386 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:35.651349    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:35.877911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:35.877871    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:36.045024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:36.044989    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:36.203808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:36.203772    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:29:36.204197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:36.203845    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:29:37.741728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:37.741696    4407 generic.go:356] "Generic (PLEG): container finished" podID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerID="9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a" exitCode=0
Nov 02 03:29:37.742270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:37.741773    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" event={"ID":"a8ea9264-44db-4610-ba04-95a8ba22a66a","Type":"ContainerDied","Data":"9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a"}
Nov 02 03:29:37.742338 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:37.742307    4407 scope.go:122] "RemoveContainer" containerID="9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a"
Nov 02 03:29:37.743893 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:37.743856    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:37.744005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:37.743928    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:37.744005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:37.743961    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:29:38.751231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:38.751193    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" event={"ID":"a8ea9264-44db-4610-ba04-95a8ba22a66a","Type":"ContainerStarted","Data":"75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7"}
Nov 02 03:29:39.229231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:39.229136    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:29:39.229586 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:39.229553    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 03:29:39.229766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:39.229610    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 03:29:40.229255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:40.229215    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 03:29:40.229713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:40.229296    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 03:29:40.747584 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:40.747530    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:40.906932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:40.906882    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:41.215206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:41.215074    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:41.229408 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:41.229373    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 03:29:41.229911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:41.229437    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 03:29:41.906979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:41.906938    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:42.229390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:42.229311    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 03:29:42.229390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:42.229355    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 03:29:42.453370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:42.453343    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:42.759198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:42.759164    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:43.229110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:43.229028    4407 patch_prober.go:29] interesting pod/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 03:29:43.229110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:43.229092    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 03:29:44.234501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:44.234462    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:29:44.234903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:44.234888    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:29:44.235574 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:44.235553    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:29:44.279082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:44.278925    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:44.457505 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:44.457464    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:44.720135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:44.720097    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"]
Nov 02 03:29:44.828938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:44.828906    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"]
Nov 02 03:29:45.100456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:45.100426    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:45.248190 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:45.248163    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:46.171318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:46.171270    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:49.096917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:49.096886    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:49.279481 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:49.279441    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:49.450748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:49.450651    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:49.721212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:49.721124    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg" podUID="15b3a067-d14c-4f00-9f35-d872a57bdd46" containerName="payload-processing" containerID="cri-o://f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4" gracePeriod=25
Nov 02 03:29:49.829861 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:49.829821    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74" podUID="20eb9978-ed21-4b9a-9c54-5652c26aa8d1" containerName="payload-pre-processing" containerID="cri-o://841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543" gracePeriod=25
Nov 02 03:29:49.999784 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:49.999759    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:29:50.109056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.109026    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:29:50.185295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.185264    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15b3a067-d14c-4f00-9f35-d872a57bdd46-kube-api-access-5xwvn\" (UniqueName: \"kubernetes.io/projected/15b3a067-d14c-4f00-9f35-d872a57bdd46-kube-api-access-5xwvn\") pod \"15b3a067-d14c-4f00-9f35-d872a57bdd46\" (UID: \"15b3a067-d14c-4f00-9f35-d872a57bdd46\") "
Nov 02 03:29:50.185508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.185466    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15b3a067-d14c-4f00-9f35-d872a57bdd46-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15b3a067-d14c-4f00-9f35-d872a57bdd46-plugins-config-volume\") pod \"15b3a067-d14c-4f00-9f35-d872a57bdd46\" (UID: \"15b3a067-d14c-4f00-9f35-d872a57bdd46\") "
Nov 02 03:29:50.185896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.185873    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15b3a067-d14c-4f00-9f35-d872a57bdd46-plugins-config-volume" pod "15b3a067-d14c-4f00-9f35-d872a57bdd46" (UID: "15b3a067-d14c-4f00-9f35-d872a57bdd46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:29:50.187539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.187514    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15b3a067-d14c-4f00-9f35-d872a57bdd46-kube-api-access-5xwvn" pod "15b3a067-d14c-4f00-9f35-d872a57bdd46" (UID: "15b3a067-d14c-4f00-9f35-d872a57bdd46"). InnerVolumeSpecName "kube-api-access-5xwvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:29:50.287060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.287032    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-kube-api-access-jsx82\" (UniqueName: \"kubernetes.io/projected/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-kube-api-access-jsx82\") pod \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\" (UID: \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\") "
Nov 02 03:29:50.287232 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.287103    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-plugins-config-volume\") pod \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\" (UID: \"20eb9978-ed21-4b9a-9c54-5652c26aa8d1\") "
Nov 02 03:29:50.287470 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.287451    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-plugins-config-volume" pod "20eb9978-ed21-4b9a-9c54-5652c26aa8d1" (UID: "20eb9978-ed21-4b9a-9c54-5652c26aa8d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:29:50.287606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.287590    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:29:50.287694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.287609    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15b3a067-d14c-4f00-9f35-d872a57bdd46-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:29:50.287694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.287641    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xwvn\" (UniqueName: \"kubernetes.io/projected/15b3a067-d14c-4f00-9f35-d872a57bdd46-kube-api-access-5xwvn\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:29:50.289365 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.289344    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-kube-api-access-jsx82" pod "20eb9978-ed21-4b9a-9c54-5652c26aa8d1" (UID: "20eb9978-ed21-4b9a-9c54-5652c26aa8d1"). InnerVolumeSpecName "kube-api-access-jsx82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:29:50.389374 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.389347    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsx82\" (UniqueName: \"kubernetes.io/projected/20eb9978-ed21-4b9a-9c54-5652c26aa8d1-kube-api-access-jsx82\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:29:50.827668 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.827615    4407 generic.go:356] "Generic (PLEG): container finished" podID="15b3a067-d14c-4f00-9f35-d872a57bdd46" containerID="f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4" exitCode=0
Nov 02 03:29:50.827859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.827688    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"
Nov 02 03:29:50.827859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.827724    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg" event={"ID":"15b3a067-d14c-4f00-9f35-d872a57bdd46","Type":"ContainerDied","Data":"f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4"}
Nov 02 03:29:50.827859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.827802    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg" event={"ID":"15b3a067-d14c-4f00-9f35-d872a57bdd46","Type":"ContainerDied","Data":"35f15b6df79753e80a30602f38db9a4c59f45300e1e5051c5ac344de544ab1df"}
Nov 02 03:29:50.827859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.827824    4407 scope.go:122] "RemoveContainer" containerID="f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4"
Nov 02 03:29:50.829226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.829141    4407 generic.go:356] "Generic (PLEG): container finished" podID="20eb9978-ed21-4b9a-9c54-5652c26aa8d1" containerID="841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543" exitCode=0
Nov 02 03:29:50.829226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.829209    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"
Nov 02 03:29:50.829350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.829228    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74" event={"ID":"20eb9978-ed21-4b9a-9c54-5652c26aa8d1","Type":"ContainerDied","Data":"841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543"}
Nov 02 03:29:50.829350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.829274    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74" event={"ID":"20eb9978-ed21-4b9a-9c54-5652c26aa8d1","Type":"ContainerDied","Data":"741749403df182d17a4bd63c838def04f470f9b7f852034aa93127050b7a115a"}
Nov 02 03:29:50.837689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.837673    4407 scope.go:122] "RemoveContainer" containerID="f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4"
Nov 02 03:29:50.837980 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:29:50.837956    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4\": container with ID starting with f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4 not found: ID does not exist" containerID="f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4"
Nov 02 03:29:50.838042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.837984    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4"} err="failed to get container status \"f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4\": rpc error: code = NotFound desc = could not find container \"f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4\": container with ID starting with f251eb24fbb0606b3e0691273c4b95f1012f2f1d3d30ba1ca761723504f586f4 not found: ID does not exist"
Nov 02 03:29:50.838042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.838002    4407 scope.go:122] "RemoveContainer" containerID="841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543"
Nov 02 03:29:50.846751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.846733    4407 scope.go:122] "RemoveContainer" containerID="841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543"
Nov 02 03:29:50.847027 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:29:50.847009    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543\": container with ID starting with 841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543 not found: ID does not exist" containerID="841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543"
Nov 02 03:29:50.847090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.847037    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543"} err="failed to get container status \"841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543\": rpc error: code = NotFound desc = could not find container \"841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543\": container with ID starting with 841f1caefa07debd8ddbb97b5e164aa2768e4eb3c1dbefa2addc2c199dd1f543 not found: ID does not exist"
Nov 02 03:29:50.993456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:50.993375    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"]
Nov 02 03:29:51.044034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.043995    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-dd87f0-5d4c94f975-qcg74"]
Nov 02 03:29:51.046684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.046665    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"]
Nov 02 03:29:51.047240 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.047228    4407 cpu_manager.go:414] "containerMap: removing container" podUID="15b3a067-d14c-4f00-9f35-d872a57bdd46" containerName="payload-processing"
Nov 02 03:29:51.047278 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.047241    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15b3a067-d14c-4f00-9f35-d872a57bdd46" containerName="payload-processing"
Nov 02 03:29:51.047278 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.047258    4407 cpu_manager.go:414] "containerMap: removing container" podUID="20eb9978-ed21-4b9a-9c54-5652c26aa8d1" containerName="payload-pre-processing"
Nov 02 03:29:51.047278 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.047263    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20eb9978-ed21-4b9a-9c54-5652c26aa8d1" containerName="payload-pre-processing"
Nov 02 03:29:51.047372 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.047351    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="15b3a067-d14c-4f00-9f35-d872a57bdd46" containerName="payload-processing"
Nov 02 03:29:51.047372 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.047362    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="20eb9978-ed21-4b9a-9c54-5652c26aa8d1" containerName="payload-pre-processing"
Nov 02 03:29:51.052088 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.052072    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.103543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.103510    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"]
Nov 02 03:29:51.104879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.104851    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-abc348-serving-cert\""
Nov 02 03:29:51.153499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.153458    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"]
Nov 02 03:29:51.156948 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.156922    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-dd87f0-5cb57977f9-hcrdg"]
Nov 02 03:29:51.199325 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.199290    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.199498 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.199447    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8t4f\" (UniqueName: \"kubernetes.io/projected/083082f5-1ba8-46ea-a8e7-e0d588a66a42-kube-api-access-p8t4f\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.199592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.199557    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-metrics-certs\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.199720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.199697    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/083082f5-1ba8-46ea-a8e7-e0d588a66a42-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.266037 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.261668    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:51.301747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.301663    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.301747 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:29:51.301737    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-serving-cert: secret "maas-api-e2e-ipp-a-abc348-serving-cert" not found
Nov 02 03:29:51.302028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.301770    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8t4f\" (UniqueName: \"kubernetes.io/projected/083082f5-1ba8-46ea-a8e7-e0d588a66a42-kube-api-access-p8t4f\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.302028 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:29:51.301815    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls podName:083082f5-1ba8-46ea-a8e7-e0d588a66a42 nodeName:}" failed. No retries permitted until 2026-11-02 03:29:51.801794329 +0000 UTC m=+2535.597142888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls") pod "maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz" (UID: "083082f5-1ba8-46ea-a8e7-e0d588a66a42") : secret "maas-api-e2e-ipp-a-abc348-serving-cert" not found
Nov 02 03:29:51.302028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.301851    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-metrics-certs\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.302028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.301924    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/083082f5-1ba8-46ea-a8e7-e0d588a66a42-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.302563 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.302538    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/083082f5-1ba8-46ea-a8e7-e0d588a66a42-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.304588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.304564    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-metrics-certs\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.359640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.359591    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8t4f\" (UniqueName: \"kubernetes.io/projected/083082f5-1ba8-46ea-a8e7-e0d588a66a42-kube-api-access-p8t4f\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.422933 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.422883    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:51.577421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.577318    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:51.808752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.808723    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.811191 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.811085    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls\") pod \"maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.966050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.965968    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:51.986777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:51.986746    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:29:52.343143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:52.343117    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"]
Nov 02 03:29:52.343996 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:29:52.343971    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod083082f5_1ba8_46ea_a8e7_e0d588a66a42.slice/crio-f7ca64450f24dac5997d985b5ccd52421f96ba6cf118ac485e0f306d3ef5be42 WatchSource:0}: Error finding container f7ca64450f24dac5997d985b5ccd52421f96ba6cf118ac485e0f306d3ef5be42: Status 404 returned error can't find the container with id f7ca64450f24dac5997d985b5ccd52421f96ba6cf118ac485e0f306d3ef5be42
Nov 02 03:29:52.352174 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:52.352148    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15b3a067-d14c-4f00-9f35-d872a57bdd46" path="/var/lib/kubelet/pods/15b3a067-d14c-4f00-9f35-d872a57bdd46/volumes"
Nov 02 03:29:52.352559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:52.352541    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20eb9978-ed21-4b9a-9c54-5652c26aa8d1" path="/var/lib/kubelet/pods/20eb9978-ed21-4b9a-9c54-5652c26aa8d1/volumes"
Nov 02 03:29:52.844830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:52.844791    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz" event={"ID":"083082f5-1ba8-46ea-a8e7-e0d588a66a42","Type":"ContainerStarted","Data":"f7ca64450f24dac5997d985b5ccd52421f96ba6cf118ac485e0f306d3ef5be42"}
Nov 02 03:29:53.852015 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:53.851981    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz" event={"ID":"083082f5-1ba8-46ea-a8e7-e0d588a66a42","Type":"ContainerStarted","Data":"efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992"}
Nov 02 03:29:53.852421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:53.852097    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:29:53.970724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:53.970669    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz" podStartSLOduration=3.13624164 podStartE2EDuration="3.970650672s" totalImagesPullingTime="834.409032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:29:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:29:53.968992875 +0000 UTC m=+2537.764341440" watchObservedRunningTime="2026-11-02 03:29:53.970650672 +0000 UTC m=+2537.765999234"
Nov 02 03:29:54.347833 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:54.347802    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:54.527064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:54.527026    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:56.597287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:56.597253    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:29:59.899022 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:29:59.898992    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:30:01.821302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:01.821275    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:05.189452 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:30:05.189411    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:30:06.776596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:06.776563    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:11.521370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:11.521331    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:13.248249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:13.248213    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:16.300863 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:16.300829    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:17.808492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:17.808456    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:30:18.785244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:18.785212    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:18.968838 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:18.968809    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:19.655882 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:19.655848    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:20.220789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:20.220750    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"]
Nov 02 03:30:20.330905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:20.330872    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"]
Nov 02 03:30:20.421888 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:20.421846    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:20.601734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:20.601698    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:20.761677 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:20.761648    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:21.161879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:21.161850    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:21.595241 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:21.595210    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:22.416410 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:22.416376    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:25.221600 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.221534    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f" podUID="3292ca90-b8ac-4ef1-98f4-4d180d777f4c" containerName="payload-processing" containerID="cri-o://c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774" gracePeriod=25
Nov 02 03:30:25.331670 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.331633    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5" podUID="ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9" containerName="payload-pre-processing" containerID="cri-o://3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d" gracePeriod=25
Nov 02 03:30:25.495543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.495514    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:30:25.570572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.570542    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-plugins-config-volume\") pod \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\" (UID: \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\") "
Nov 02 03:30:25.570835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.570818    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-kube-api-access-p664b\" (UniqueName: \"kubernetes.io/projected/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-kube-api-access-p664b\") pod \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\" (UID: \"3292ca90-b8ac-4ef1-98f4-4d180d777f4c\") "
Nov 02 03:30:25.571047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.571019    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-plugins-config-volume" pod "3292ca90-b8ac-4ef1-98f4-4d180d777f4c" (UID: "3292ca90-b8ac-4ef1-98f4-4d180d777f4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:30:25.571353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.571326    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:30:25.573799 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.573773    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-kube-api-access-p664b" pod "3292ca90-b8ac-4ef1-98f4-4d180d777f4c" (UID: "3292ca90-b8ac-4ef1-98f4-4d180d777f4c"). InnerVolumeSpecName "kube-api-access-p664b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:30:25.611115 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.611074    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:30:25.672316 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.672282    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-plugins-config-volume\") pod \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\" (UID: \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\") "
Nov 02 03:30:25.672482 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.672395    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-kube-api-access-xwj5d\" (UniqueName: \"kubernetes.io/projected/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-kube-api-access-xwj5d\") pod \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\" (UID: \"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9\") "
Nov 02 03:30:25.672902 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.672878    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-plugins-config-volume" pod "ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9" (UID: "ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:30:25.673091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.673072    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:30:25.673136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.673099    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p664b\" (UniqueName: \"kubernetes.io/projected/3292ca90-b8ac-4ef1-98f4-4d180d777f4c-kube-api-access-p664b\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:30:25.674990 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.674964    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-kube-api-access-xwj5d" pod "ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9" (UID: "ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9"). InnerVolumeSpecName "kube-api-access-xwj5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:30:25.773995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:25.773963    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwj5d\" (UniqueName: \"kubernetes.io/projected/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9-kube-api-access-xwj5d\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:30:26.143118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.143035    4407 generic.go:356] "Generic (PLEG): container finished" podID="3292ca90-b8ac-4ef1-98f4-4d180d777f4c" containerID="c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774" exitCode=0
Nov 02 03:30:26.143118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.143082    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"
Nov 02 03:30:26.143327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.143121    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f" event={"ID":"3292ca90-b8ac-4ef1-98f4-4d180d777f4c","Type":"ContainerDied","Data":"c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774"}
Nov 02 03:30:26.143327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.143204    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f" event={"ID":"3292ca90-b8ac-4ef1-98f4-4d180d777f4c","Type":"ContainerDied","Data":"fec984d0ffa92ed37da34d6d32e8dcca4e90c3ff773d73df06334ff3b7692644"}
Nov 02 03:30:26.143327 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.143233    4407 scope.go:122] "RemoveContainer" containerID="c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774"
Nov 02 03:30:26.144500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.144480    4407 generic.go:356] "Generic (PLEG): container finished" podID="ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9" containerID="3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d" exitCode=0
Nov 02 03:30:26.144572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.144529    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5" event={"ID":"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9","Type":"ContainerDied","Data":"3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d"}
Nov 02 03:30:26.144572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.144545    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"
Nov 02 03:30:26.144572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.144567    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5" event={"ID":"ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9","Type":"ContainerDied","Data":"11be9dee2a3a9635aac9d0fa8b83ea62095b94fd367b6aedb4bb1e43988026a4"}
Nov 02 03:30:26.153258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.153240    4407 scope.go:122] "RemoveContainer" containerID="c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774"
Nov 02 03:30:26.153637 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:30:26.153601    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774\": container with ID starting with c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774 not found: ID does not exist" containerID="c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774"
Nov 02 03:30:26.153725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.153647    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774"} err="failed to get container status \"c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774\": rpc error: code = NotFound desc = could not find container \"c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774\": container with ID starting with c534092edfc811847d0883f6a543696e29a55d7af085c5a6ddb0318c835fd774 not found: ID does not exist"
Nov 02 03:30:26.153725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.153666    4407 scope.go:122] "RemoveContainer" containerID="3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d"
Nov 02 03:30:26.163405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.163384    4407 scope.go:122] "RemoveContainer" containerID="3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d"
Nov 02 03:30:26.163710 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:30:26.163690    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d\": container with ID starting with 3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d not found: ID does not exist" containerID="3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d"
Nov 02 03:30:26.163808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.163719    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d"} err="failed to get container status \"3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d\": rpc error: code = NotFound desc = could not find container \"3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d\": container with ID starting with 3c651559de169bf7ed811aadccb2263f0f8ee851b89cbaf786ccfd91e155c87d not found: ID does not exist"
Nov 02 03:30:26.311440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.311403    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"]
Nov 02 03:30:26.314255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.314226    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-045653-8598bc489f-fhdw5"]
Nov 02 03:30:26.351352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.351312    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9" path="/var/lib/kubelet/pods/ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9/volumes"
Nov 02 03:30:26.424260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.424043    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"]
Nov 02 03:30:26.428666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.428638    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-045653-594b8444f9-crb5f"]
Nov 02 03:30:26.767300 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:26.767253    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:27.034661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:27.034567    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:27.191104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:27.191064    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:28.014347 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:28.014308    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:28.352977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:28.352899    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3292ca90-b8ac-4ef1-98f4-4d180d777f4c" path="/var/lib/kubelet/pods/3292ca90-b8ac-4ef1-98f4-4d180d777f4c/volumes"
Nov 02 03:30:30.058432 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:30.058385    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:30.221038 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:30.221007    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:33.152833 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:33.152802    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:35.193088 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:30:35.193043    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:30:37.793945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.793915    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"]
Nov 02 03:30:37.794473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.794460    4407 cpu_manager.go:414] "containerMap: removing container" podUID="ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9" containerName="payload-pre-processing"
Nov 02 03:30:37.794528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.794474    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9" containerName="payload-pre-processing"
Nov 02 03:30:37.794528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.794510    4407 cpu_manager.go:414] "containerMap: removing container" podUID="3292ca90-b8ac-4ef1-98f4-4d180d777f4c" containerName="payload-processing"
Nov 02 03:30:37.794528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.794516    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3292ca90-b8ac-4ef1-98f4-4d180d777f4c" containerName="payload-processing"
Nov 02 03:30:37.794642 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.794604    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="3292ca90-b8ac-4ef1-98f4-4d180d777f4c" containerName="payload-processing"
Nov 02 03:30:37.794642 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.794615    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee2ad395-ff5b-4b7e-bbf2-eac5e68648f9" containerName="payload-pre-processing"
Nov 02 03:30:37.798981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.798965    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:37.853912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.853883    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"]
Nov 02 03:30:37.901054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.901026    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-d94cc3-serving-cert\""
Nov 02 03:30:37.928900 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.928852    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:37.929194 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.929173    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf9b24a-128d-4247-92c0-e8b70113e800-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:37.929373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.929307    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbzwc\" (UniqueName: \"kubernetes.io/projected/ecf9b24a-128d-4247-92c0-e8b70113e800-kube-api-access-dbzwc\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:37.929373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.929345    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:37.999867 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:37.999829    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:38.030175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.030142    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.030371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.030222    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf9b24a-128d-4247-92c0-e8b70113e800-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.030371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.030271    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbzwc\" (UniqueName: \"kubernetes.io/projected/ecf9b24a-128d-4247-92c0-e8b70113e800-kube-api-access-dbzwc\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.030371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.030294    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.030371 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:30:38.030311    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-serving-cert: secret "maas-api-e2e-shared-b-w2-d94cc3-serving-cert" not found
Nov 02 03:30:38.030583 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:30:38.030394    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls podName:ecf9b24a-128d-4247-92c0-e8b70113e800 nodeName:}" failed. No retries permitted until 2026-11-02 03:30:38.530374751 +0000 UTC m=+2582.325723296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls") pod "maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c" (UID: "ecf9b24a-128d-4247-92c0-e8b70113e800") : secret "maas-api-e2e-shared-b-w2-d94cc3-serving-cert" not found
Nov 02 03:30:38.031078 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.030990    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf9b24a-128d-4247-92c0-e8b70113e800-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.032911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.032885    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.090936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.090863    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbzwc\" (UniqueName: \"kubernetes.io/projected/ecf9b24a-128d-4247-92c0-e8b70113e800-kube-api-access-dbzwc\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.156037 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.156005    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:38.339743 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.339707    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:38.536088 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.536061    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.538594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.538570    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:38.712196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:38.712161    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:39.091606 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:30:39.091568    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf9b24a_128d_4247_92c0_e8b70113e800.slice/crio-6d105e69439d3f36da75fc9909020dafc8ee9799958ec42a1e39f7d8d10379e9 WatchSource:0}: Error finding container 6d105e69439d3f36da75fc9909020dafc8ee9799958ec42a1e39f7d8d10379e9: Status 404 returned error can't find the container with id 6d105e69439d3f36da75fc9909020dafc8ee9799958ec42a1e39f7d8d10379e9
Nov 02 03:30:39.092064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:39.091975    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"]
Nov 02 03:30:39.271890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:39.271859    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c" event={"ID":"ecf9b24a-128d-4247-92c0-e8b70113e800","Type":"ContainerStarted","Data":"6d105e69439d3f36da75fc9909020dafc8ee9799958ec42a1e39f7d8d10379e9"}
Nov 02 03:30:40.279159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:40.279124    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c" event={"ID":"ecf9b24a-128d-4247-92c0-e8b70113e800","Type":"ContainerStarted","Data":"c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6"}
Nov 02 03:30:40.279617 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:40.279181    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:40.375425 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:40.375395    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:40.399889 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:40.399818    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c" podStartSLOduration=2.503522519 podStartE2EDuration="3.399795865s" totalImagesPullingTime="896.273346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:30:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:30:40.394577891 +0000 UTC m=+2584.189926458" watchObservedRunningTime="2026-11-02 03:30:40.399795865 +0000 UTC m=+2584.195144438"
Nov 02 03:30:40.523447 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:40.523412    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:42.790021 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:42.789983    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:43.218698 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:43.218608    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:46.291028 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:46.291000    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:30:46.767275 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:46.767248    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:47.286270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:47.286235    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:47.562532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:47.562441    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:47.732195 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:47.732125    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:49.861737 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:49.861698    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:54.664763 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:54.664728    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:55.382575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:55.382544    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:59.465815 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.465785    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"]
Nov 02 03:30:59.470082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.470065    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.479661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.479617    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"]
Nov 02 03:30:59.514224 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.514193    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-metrics-certs\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.514385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.514241    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.514430 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.514405    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjfpd\" (UniqueName: \"kubernetes.io/projected/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-kube-api-access-bjfpd\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.514559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.514538    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.526095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.526068    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-018375-serving-cert\""
Nov 02 03:30:59.529777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.529751    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:30:59.616078 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.616046    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-metrics-certs\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.616270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.616092    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.616270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.616147    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjfpd\" (UniqueName: \"kubernetes.io/projected/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-kube-api-access-bjfpd\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.616270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.616204    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.616270 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:30:59.616241    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-serving-cert: secret "maas-api-e2e-ipp-b-018375-serving-cert" not found
Nov 02 03:30:59.616471 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:30:59.616319    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls podName:32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a nodeName:}" failed. No retries permitted until 2026-11-02 03:31:00.116300392 +0000 UTC m=+2603.911648959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls") pod "maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb" (UID: "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a") : secret "maas-api-e2e-ipp-b-018375-serving-cert" not found
Nov 02 03:30:59.616940 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.616917    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.618747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.618728    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-metrics-certs\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.626037 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.626007    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:59.675994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.675963    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjfpd\" (UniqueName: \"kubernetes.io/projected/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-kube-api-access-bjfpd\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:30:59.805823 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.805791    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:30:59.991462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:30:59.991428    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:00.126436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:00.126337    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:31:00.129110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:00.129086    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:31:00.300835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:00.300801    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:00.388974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:00.388880    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:31:00.816873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:00.816840    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"]
Nov 02 03:31:00.817319 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:31:00.817297    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32bb14ba_5bb5_4651_ad64_4c9d93bb4b1a.slice/crio-c038384789f1e11e0a4c335e0e980c9e39866e791adae927a048348234ca4313 WatchSource:0}: Error finding container c038384789f1e11e0a4c335e0e980c9e39866e791adae927a048348234ca4313: Status 404 returned error can't find the container with id c038384789f1e11e0a4c335e0e980c9e39866e791adae927a048348234ca4313
Nov 02 03:31:01.452636 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:01.452601    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb" event={"ID":"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a","Type":"ContainerStarted","Data":"c038384789f1e11e0a4c335e0e980c9e39866e791adae927a048348234ca4313"}
Nov 02 03:31:02.479678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:02.479612    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb" event={"ID":"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a","Type":"ContainerStarted","Data":"29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946"}
Nov 02 03:31:02.486162 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:02.486136    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:31:02.606288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:02.606223    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb" podStartSLOduration=2.7841438419999998 podStartE2EDuration="3.606203012s" totalImagesPullingTime="822.05917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:31:02.604818329 +0000 UTC m=+2606.400166895" watchObservedRunningTime="2026-11-02 03:31:02.606203012 +0000 UTC m=+2606.401551578"
Nov 02 03:31:04.809973 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:04.809942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:05.215472 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:31:05.215386    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:31:09.520306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:09.520271    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:31:10.107264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:10.107233    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:10.552411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:10.552380    4407 generic.go:356] "Generic (PLEG): container finished" podID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerID="6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd" exitCode=0
Nov 02 03:31:10.552952 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:10.552458    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" event={"ID":"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3","Type":"ContainerDied","Data":"6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd"}
Nov 02 03:31:10.553037 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:10.553018    4407 scope.go:122] "RemoveContainer" containerID="6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd"
Nov 02 03:31:10.554684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:10.554659    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:31:10.554764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:10.554722    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:31:10.554764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:10.554752    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:31:10.979138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:10.979058    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:11.560117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:11.560085    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" event={"ID":"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3","Type":"ContainerStarted","Data":"f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e"}
Nov 02 03:31:11.606649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:11.606582    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"]
Nov 02 03:31:11.710752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:11.710716    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"]
Nov 02 03:31:11.810222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:11.810094    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:11.964320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:11.964281    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:12.127088 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:12.127006    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:12.153199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:12.153170    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:31:12.154504 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:12.154477    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 02 03:31:12.154680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:12.154533    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 02 03:31:12.434713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:12.432852    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:12.611843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:12.610383    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:13.152912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:13.152886    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 02 03:31:13.153149 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:13.152988    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 02 03:31:14.153725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:14.153683    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 02 03:31:14.154208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:14.153757    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 02 03:31:14.449482 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:14.449371    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:14.614801 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:14.612904    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:15.153440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:15.153408    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 02 03:31:15.153665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:15.153463    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 02 03:31:15.518547 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:15.518513    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:15.672759 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:15.672728    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:31:16.152715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:16.152678    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 02 03:31:16.152929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:16.152740    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 02 03:31:16.606968 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:16.606906    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq" podUID="35b2436f-d6f4-427e-994d-4515e582bcde" containerName="payload-processing" containerID="cri-o://a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db" gracePeriod=25
Nov 02 03:31:16.711612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:16.711548    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl" podUID="0dce5475-f780-4807-b644-0914f37810a6" containerName="payload-pre-processing" containerID="cri-o://fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0" gracePeriod=25
Nov 02 03:31:16.900877 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:16.900848    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:31:17.007268 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.007238    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:31:17.008364 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.008341    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:17.066616 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.066582    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35b2436f-d6f4-427e-994d-4515e582bcde-kube-api-access-bbbbw\" (UniqueName: \"kubernetes.io/projected/35b2436f-d6f4-427e-994d-4515e582bcde-kube-api-access-bbbbw\") pod \"35b2436f-d6f4-427e-994d-4515e582bcde\" (UID: \"35b2436f-d6f4-427e-994d-4515e582bcde\") "
Nov 02 03:31:17.066932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.066910    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35b2436f-d6f4-427e-994d-4515e582bcde-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35b2436f-d6f4-427e-994d-4515e582bcde-plugins-config-volume\") pod \"35b2436f-d6f4-427e-994d-4515e582bcde\" (UID: \"35b2436f-d6f4-427e-994d-4515e582bcde\") "
Nov 02 03:31:17.067308 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.067281    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35b2436f-d6f4-427e-994d-4515e582bcde-plugins-config-volume" pod "35b2436f-d6f4-427e-994d-4515e582bcde" (UID: "35b2436f-d6f4-427e-994d-4515e582bcde"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:31:17.067954 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.067932    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35b2436f-d6f4-427e-994d-4515e582bcde-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:31:17.071752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.071724    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b2436f-d6f4-427e-994d-4515e582bcde-kube-api-access-bbbbw" pod "35b2436f-d6f4-427e-994d-4515e582bcde" (UID: "35b2436f-d6f4-427e-994d-4515e582bcde"). InnerVolumeSpecName "kube-api-access-bbbbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:31:17.168570 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.168490    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0dce5475-f780-4807-b644-0914f37810a6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dce5475-f780-4807-b644-0914f37810a6-plugins-config-volume\") pod \"0dce5475-f780-4807-b644-0914f37810a6\" (UID: \"0dce5475-f780-4807-b644-0914f37810a6\") "
Nov 02 03:31:17.168770 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.168750    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dce5475-f780-4807-b644-0914f37810a6-kube-api-access-w8d6p\" (UniqueName: \"kubernetes.io/projected/0dce5475-f780-4807-b644-0914f37810a6-kube-api-access-w8d6p\") pod \"0dce5475-f780-4807-b644-0914f37810a6\" (UID: \"0dce5475-f780-4807-b644-0914f37810a6\") "
Nov 02 03:31:17.168977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.168955    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dce5475-f780-4807-b644-0914f37810a6-plugins-config-volume" pod "0dce5475-f780-4807-b644-0914f37810a6" (UID: "0dce5475-f780-4807-b644-0914f37810a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:31:17.169307 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.169288    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbbbw\" (UniqueName: \"kubernetes.io/projected/35b2436f-d6f4-427e-994d-4515e582bcde-kube-api-access-bbbbw\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:31:17.169583 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.169310    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dce5475-f780-4807-b644-0914f37810a6-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:31:17.171013 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.170984    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dce5475-f780-4807-b644-0914f37810a6-kube-api-access-w8d6p" pod "0dce5475-f780-4807-b644-0914f37810a6" (UID: "0dce5475-f780-4807-b644-0914f37810a6"). InnerVolumeSpecName "kube-api-access-w8d6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:31:17.205080 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.205047    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:17.238990 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.238961    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:31:17.240549 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.240527    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 03:31:17.270618 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.270588    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8d6p\" (UniqueName: \"kubernetes.io/projected/0dce5475-f780-4807-b644-0914f37810a6-kube-api-access-w8d6p\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:31:17.377101 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.377075    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:17.621463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.621429    4407 generic.go:356] "Generic (PLEG): container finished" podID="35b2436f-d6f4-427e-994d-4515e582bcde" containerID="a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db" exitCode=0
Nov 02 03:31:17.621971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.621486    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"
Nov 02 03:31:17.621971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.621519    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq" event={"ID":"35b2436f-d6f4-427e-994d-4515e582bcde","Type":"ContainerDied","Data":"a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db"}
Nov 02 03:31:17.621971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.621570    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq" event={"ID":"35b2436f-d6f4-427e-994d-4515e582bcde","Type":"ContainerDied","Data":"73076414e569ce0a1785fb24b758bf9302fb967affdfc7dacabdd30e1fa74d9d"}
Nov 02 03:31:17.621971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.621601    4407 scope.go:122] "RemoveContainer" containerID="a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db"
Nov 02 03:31:17.623123 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.623100    4407 generic.go:356] "Generic (PLEG): container finished" podID="0dce5475-f780-4807-b644-0914f37810a6" containerID="fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0" exitCode=0
Nov 02 03:31:17.623266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.623151    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl" event={"ID":"0dce5475-f780-4807-b644-0914f37810a6","Type":"ContainerDied","Data":"fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0"}
Nov 02 03:31:17.623266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.623197    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl" event={"ID":"0dce5475-f780-4807-b644-0914f37810a6","Type":"ContainerDied","Data":"57fbbd66ac0d5007fb52752974438a195e64c7a9a89e2db2b6abfe5d5ecc593b"}
Nov 02 03:31:17.623557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.623373    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"
Nov 02 03:31:17.637957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.637929    4407 scope.go:122] "RemoveContainer" containerID="a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db"
Nov 02 03:31:17.638342 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:31:17.638310    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db\": container with ID starting with a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db not found: ID does not exist" containerID="a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db"
Nov 02 03:31:17.638415 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.638356    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db"} err="failed to get container status \"a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db\": rpc error: code = NotFound desc = could not find container \"a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db\": container with ID starting with a27f64b7308d2e96898a9ea463c8a6ef053cf97a7c6a55d28a64babbd9a377db not found: ID does not exist"
Nov 02 03:31:17.638415 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.638382    4407 scope.go:122] "RemoveContainer" containerID="fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0"
Nov 02 03:31:17.721484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.721457    4407 scope.go:122] "RemoveContainer" containerID="fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0"
Nov 02 03:31:17.721964 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:31:17.721936    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0\": container with ID starting with fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0 not found: ID does not exist" containerID="fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0"
Nov 02 03:31:17.722094 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.721976    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0"} err="failed to get container status \"fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0\": rpc error: code = NotFound desc = could not find container \"fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0\": container with ID starting with fef626da1f3402db76cf18d673d53b6899e3fb2c92c333ffbd08a262b1a028d0 not found: ID does not exist"
Nov 02 03:31:17.825370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.825332    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"]
Nov 02 03:31:17.832236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.832207    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-abc348-765fc6594c-276rq"]
Nov 02 03:31:17.942674 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.942565    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"]
Nov 02 03:31:17.946967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:17.946939    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-abc348-78b6df7b67-bsvdl"]
Nov 02 03:31:18.354401 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:18.354369    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0dce5475-f780-4807-b644-0914f37810a6" path="/var/lib/kubelet/pods/0dce5475-f780-4807-b644-0914f37810a6/volumes"
Nov 02 03:31:18.354860 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:18.354844    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35b2436f-d6f4-427e-994d-4515e582bcde" path="/var/lib/kubelet/pods/35b2436f-d6f4-427e-994d-4515e582bcde/volumes"
Nov 02 03:31:20.240854 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.240806    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:20.410402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.410363    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:20.629184 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.629155    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"]
Nov 02 03:31:20.629756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.629742    4407 cpu_manager.go:414] "containerMap: removing container" podUID="35b2436f-d6f4-427e-994d-4515e582bcde" containerName="payload-processing"
Nov 02 03:31:20.629822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.629759    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35b2436f-d6f4-427e-994d-4515e582bcde" containerName="payload-processing"
Nov 02 03:31:20.629822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.629790    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0dce5475-f780-4807-b644-0914f37810a6" containerName="payload-pre-processing"
Nov 02 03:31:20.629822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.629795    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dce5475-f780-4807-b644-0914f37810a6" containerName="payload-pre-processing"
Nov 02 03:31:20.629927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.629886    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="35b2436f-d6f4-427e-994d-4515e582bcde" containerName="payload-processing"
Nov 02 03:31:20.629927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.629898    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dce5475-f780-4807-b644-0914f37810a6" containerName="payload-pre-processing"
Nov 02 03:31:20.634371 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.634354    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:20.691696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.691663    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"]
Nov 02 03:31:20.749648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.742878    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0809c8\""
Nov 02 03:31:20.749648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.743525    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-0809c8-dockercfg-5d9dt\""
Nov 02 03:31:20.816680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.816607    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvlw6\" (UniqueName: \"kubernetes.io/projected/c714449c-4518-43a2-b7dd-cdd4a6b64791-kube-api-access-vvlw6\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt\" (UID: \"c714449c-4518-43a2-b7dd-cdd4a6b64791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:20.816858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.816776    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c714449c-4518-43a2-b7dd-cdd4a6b64791-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt\" (UID: \"c714449c-4518-43a2-b7dd-cdd4a6b64791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:20.835703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.835671    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:20.918271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.918165    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvlw6\" (UniqueName: \"kubernetes.io/projected/c714449c-4518-43a2-b7dd-cdd4a6b64791-kube-api-access-vvlw6\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt\" (UID: \"c714449c-4518-43a2-b7dd-cdd4a6b64791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:20.918271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.918232    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c714449c-4518-43a2-b7dd-cdd4a6b64791-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt\" (UID: \"c714449c-4518-43a2-b7dd-cdd4a6b64791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:20.919087 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.919057    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c714449c-4518-43a2-b7dd-cdd4a6b64791-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt\" (UID: \"c714449c-4518-43a2-b7dd-cdd4a6b64791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:20.975826 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:20.975793    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvlw6\" (UniqueName: \"kubernetes.io/projected/c714449c-4518-43a2-b7dd-cdd4a6b64791-kube-api-access-vvlw6\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt\" (UID: \"c714449c-4518-43a2-b7dd-cdd4a6b64791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:21.248400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:21.248372    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:21.629820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:21.629792    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"]
Nov 02 03:31:21.630916 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:31:21.630893    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc714449c_4518_43a2_b7dd_cdd4a6b64791.slice/crio-9f894110477f12b022594d8cefdb7ab015911dc412705ff1b98bc8a0d35337f0 WatchSource:0}: Error finding container 9f894110477f12b022594d8cefdb7ab015911dc412705ff1b98bc8a0d35337f0: Status 404 returned error can't find the container with id 9f894110477f12b022594d8cefdb7ab015911dc412705ff1b98bc8a0d35337f0
Nov 02 03:31:21.650526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:21.650502    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt" event={"ID":"c714449c-4518-43a2-b7dd-cdd4a6b64791","Type":"ContainerStarted","Data":"9f894110477f12b022594d8cefdb7ab015911dc412705ff1b98bc8a0d35337f0"}
Nov 02 03:31:22.660691 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:22.660648    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt" event={"ID":"c714449c-4518-43a2-b7dd-cdd4a6b64791","Type":"ContainerStarted","Data":"203df744412fed47128636df1c72dfddc97d194485a73cc31cf132727d54ffff"}
Nov 02 03:31:22.661090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:22.660722    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:22.778533 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:22.778480    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt" podStartSLOduration=2.778466049 podStartE2EDuration="2.778466049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:31:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:31:22.77561199 +0000 UTC m=+2626.570960556" watchObservedRunningTime="2026-11-02 03:31:22.778466049 +0000 UTC m=+2626.573814650"
Nov 02 03:31:26.363014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:26.362972    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:30.135528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:30.135496    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:30.646147 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:30.646115    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:33.510273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:33.510239    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:33.670055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:33.670023    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:31:35.189153 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:31:35.189113    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:31:38.377905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:38.377874    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:39.839126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:39.839096    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:31:43.057440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:43.057405    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:43.209094 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:43.209066    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:43.365750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:43.365656    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:48.056352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:48.056271    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:52.791936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:52.791895    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:31:57.826979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:31:57.826942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:00.600756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:00.600728    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:04.705222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:04.705190    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:05.106709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:05.106677    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:05.196840 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:32:05.196800    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:32:09.506953 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:09.506916    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:14.344858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:14.344823    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:19.322316 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:19.322282    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:21.589444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:21.589412    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:23.206083 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:23.206058    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:23.514726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:23.514692    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:23.668803 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:23.668768    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:23.827855 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:23.827768    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:26.661233 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:26.661191    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:27.420424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:27.420388    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:32:31.760588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:31.760556    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:35.190256 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:32:35.190221    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:32:36.257707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.257677    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"]
Nov 02 03:32:36.266296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.266264    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.319289 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.319254    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"]
Nov 02 03:32:36.319944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.319917    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rl8m\" (UniqueName: \"kubernetes.io/projected/aa21aeaa-e46e-4090-b212-1d0bc2f608db-kube-api-access-9rl8m\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.320196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.320177    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-metrics-certs\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.320353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.320331    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.320402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.320385    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa21aeaa-e46e-4090-b212-1d0bc2f608db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.365253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.365222    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-0809c8-serving-cert\""
Nov 02 03:32:36.422038 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.422006    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rl8m\" (UniqueName: \"kubernetes.io/projected/aa21aeaa-e46e-4090-b212-1d0bc2f608db-kube-api-access-9rl8m\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.422220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.422086    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-metrics-certs\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.422220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.422196    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.422345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.422235    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa21aeaa-e46e-4090-b212-1d0bc2f608db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.422411 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:32:36.422348    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-serving-cert: secret "maas-api-e2e-ipp-b-0809c8-serving-cert" not found
Nov 02 03:32:36.422469 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:32:36.422432    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls podName:aa21aeaa-e46e-4090-b212-1d0bc2f608db nodeName:}" failed. No retries permitted until 2026-11-02 03:32:36.922409135 +0000 UTC m=+2700.717757877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls") pod "maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7" (UID: "aa21aeaa-e46e-4090-b212-1d0bc2f608db") : secret "maas-api-e2e-ipp-b-0809c8-serving-cert" not found
Nov 02 03:32:36.423003 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.422985    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa21aeaa-e46e-4090-b212-1d0bc2f608db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.424864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.424817    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-metrics-certs\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.479860 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.479829    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rl8m\" (UniqueName: \"kubernetes.io/projected/aa21aeaa-e46e-4090-b212-1d0bc2f608db-kube-api-access-9rl8m\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.481525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.481442    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:36.642314 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.642230    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:36.796533 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.796501    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:36.937939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.937840    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:36.940406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:36.940377    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:37.179228 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:37.179194    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:37.562684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:37.562653    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"]
Nov 02 03:32:37.563114 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:32:37.562793    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa21aeaa_e46e_4090_b212_1d0bc2f608db.slice/crio-4cea1248735746c87ae3ca50f82d1ce78c1c4de4a8782dea6a1dc6ecaa2faa57 WatchSource:0}: Error finding container 4cea1248735746c87ae3ca50f82d1ce78c1c4de4a8782dea6a1dc6ecaa2faa57: Status 404 returned error can't find the container with id 4cea1248735746c87ae3ca50f82d1ce78c1c4de4a8782dea6a1dc6ecaa2faa57
Nov 02 03:32:38.135964 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:38.135933    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7" event={"ID":"aa21aeaa-e46e-4090-b212-1d0bc2f608db","Type":"ContainerStarted","Data":"4cea1248735746c87ae3ca50f82d1ce78c1c4de4a8782dea6a1dc6ecaa2faa57"}
Nov 02 03:32:39.144177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:39.144136    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7" event={"ID":"aa21aeaa-e46e-4090-b212-1d0bc2f608db","Type":"ContainerStarted","Data":"df60963b175a3fa387e72a2dd58d98e10eba1b789225250490872a5721d59c07"}
Nov 02 03:32:39.144571 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:39.144290    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:39.261870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:39.261817    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7" podStartSLOduration=2.46044779 podStartE2EDuration="3.261805026s" totalImagesPullingTime="801.357236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:32:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:32:39.261413796 +0000 UTC m=+2703.056762361" watchObservedRunningTime="2026-11-02 03:32:39.261805026 +0000 UTC m=+2703.057153591"
Nov 02 03:32:41.449599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:41.449554    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:43.482257 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:43.482219    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:45.156159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:45.156128    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:32:46.388920 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:46.388882    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:52.833268 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:52.833233    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:32:57.597863 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:32:57.597833    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:02.342831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:02.342783    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:02.381815 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:02.381781    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:33:05.188162 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:33:05.188129    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:33:07.373741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:07.373712    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:10.081241 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:10.081208    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:11.110667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:11.110612    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:11.541060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:11.541028    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:11.697227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:11.697185    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:12.066574 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:12.066528    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:13.694974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:13.694938    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:13.963235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:13.963126    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:14.119086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:14.119049    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:14.481788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:14.481756    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"]
Nov 02 03:33:14.481979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:14.481954    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" containerID="cri-o://467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528" gracePeriod=30
Nov 02 03:33:14.534741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:14.534701    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"]
Nov 02 03:33:14.535008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:14.534982    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy" containerID="cri-o://3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6" gracePeriod=30
Nov 02 03:33:14.782760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:14.782732    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:15.701032 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:15.700998    4407 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-gnf96 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:33:15.701241 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:15.701052    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:33:18.754859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:18.754777    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:19.334216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.334185    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:19.806447 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.806420    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:33:19.839764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.839744    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:33:19.907407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907326    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/300055ac-05f3-4374-ab56-6d62f61699b5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/300055ac-05f3-4374-ab56-6d62f61699b5-istiod-ca-cert\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.907407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907381    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-podinfo\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.907663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907429    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-socket\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.907663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907475    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-envoy\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.907663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907525    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-data\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.907663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907566    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-kube-api-access-8npnh\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-kube-api-access-8npnh\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.907663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907601    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-envoy\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.907936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907682    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-kube-api-access-x89n4\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-kube-api-access-x89n4\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.907936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907791    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-credential-socket\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.907936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907859    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/300055ac-05f3-4374-ab56-6d62f61699b5-istiod-ca-cert" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:33:19.907936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907859    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-data" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.907936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907873    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14b6aabd-8241-47d5-93f2-494bfcef0106-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b6aabd-8241-47d5-93f2-494bfcef0106-istiod-ca-cert\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.908157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.907945    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/300055ac-05f3-4374-ab56-6d62f61699b5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/300055ac-05f3-4374-ab56-6d62f61699b5-istio-podinfo\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.908157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908010    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-credential-socket\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.908157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908031    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-socket" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.908157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908042    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-data\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.908157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908084    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-socket\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.908445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908235    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-istio-token\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-istio-token\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.908445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908288    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-certs\") pod \"300055ac-05f3-4374-ab56-6d62f61699b5\" (UID: \"300055ac-05f3-4374-ab56-6d62f61699b5\") "
Nov 02 03:33:19.908445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908319    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-certs\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.908445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908348    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-token\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-token\") pod \"14b6aabd-8241-47d5-93f2-494bfcef0106\" (UID: \"14b6aabd-8241-47d5-93f2-494bfcef0106\") "
Nov 02 03:33:19.908445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908360    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-credential-socket" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.908445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908269    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-credential-socket" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.908800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.908452    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b6aabd-8241-47d5-93f2-494bfcef0106-istiod-ca-cert" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:33:19.909210 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909140    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-certs" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.909210 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909194    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b6aabd-8241-47d5-93f2-494bfcef0106-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:19.909385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909217    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:19.909385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909233    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/300055ac-05f3-4374-ab56-6d62f61699b5-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:19.909385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909247    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:19.909385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909261    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:19.909385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909196    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-certs" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.909385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909254    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-socket" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.909385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909276    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:19.909385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.909363    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-data" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.910685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.910612    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-kube-api-access-x89n4" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "kube-api-access-x89n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:33:19.910846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.910704    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-envoy" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.910921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.910898    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-podinfo" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:33:19.910921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.910900    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/300055ac-05f3-4374-ab56-6d62f61699b5-istio-podinfo" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:33:19.911157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.911140    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-kube-api-access-8npnh" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "kube-api-access-8npnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:33:19.911660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.911608    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-envoy" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:33:19.911755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.911661    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-token" pod "14b6aabd-8241-47d5-93f2-494bfcef0106" (UID: "14b6aabd-8241-47d5-93f2-494bfcef0106"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:33:19.912240 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:19.912221    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-istio-token" pod "300055ac-05f3-4374-ab56-6d62f61699b5" (UID: "300055ac-05f3-4374-ab56-6d62f61699b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:33:20.010517 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010482    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010523    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8npnh\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-kube-api-access-8npnh\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010539    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010553    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x89n4\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-kube-api-access-x89n4\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010566    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/300055ac-05f3-4374-ab56-6d62f61699b5-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010578    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010592    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010605    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/300055ac-05f3-4374-ab56-6d62f61699b5-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010618    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/300055ac-05f3-4374-ab56-6d62f61699b5-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010659    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b6aabd-8241-47d5-93f2-494bfcef0106-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010670    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.010712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.010682    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b6aabd-8241-47d5-93f2-494bfcef0106-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:20.399382 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.399351    4407 generic.go:356] "Generic (PLEG): container finished" podID="300055ac-05f3-4374-ab56-6d62f61699b5" containerID="3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6" exitCode=0
Nov 02 03:33:20.399552 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.399392    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"
Nov 02 03:33:20.399552 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.399432    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" event={"ID":"300055ac-05f3-4374-ab56-6d62f61699b5","Type":"ContainerDied","Data":"3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6"}
Nov 02 03:33:20.399552 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.399500    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh" event={"ID":"300055ac-05f3-4374-ab56-6d62f61699b5","Type":"ContainerDied","Data":"f92e18a2b1dcf0439e3406400eb3dd74a6379db507f0500223c857b6de8e6b00"}
Nov 02 03:33:20.399552 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.399532    4407 scope.go:122] "RemoveContainer" containerID="3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6"
Nov 02 03:33:20.403664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.401538    4407 generic.go:356] "Generic (PLEG): container finished" podID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerID="467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528" exitCode=0
Nov 02 03:33:20.403664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.401584    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" event={"ID":"14b6aabd-8241-47d5-93f2-494bfcef0106","Type":"ContainerDied","Data":"467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528"}
Nov 02 03:33:20.403664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.401753    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96" event={"ID":"14b6aabd-8241-47d5-93f2-494bfcef0106","Type":"ContainerDied","Data":"56466993b3905767684d366f02f19151617d115a0baa93c69c7547cd9ca50d92"}
Nov 02 03:33:20.403664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.402253    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"
Nov 02 03:33:20.413517 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.413490    4407 scope.go:122] "RemoveContainer" containerID="3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6"
Nov 02 03:33:20.413897 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:33:20.413870    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6\": container with ID starting with 3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6 not found: ID does not exist" containerID="3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6"
Nov 02 03:33:20.414008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.413909    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6"} err="failed to get container status \"3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6\": rpc error: code = NotFound desc = could not find container \"3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6\": container with ID starting with 3ef02ee970d46a76ab24a4ced82014052c06e8933a4a8e80379c57e7d7c35db6 not found: ID does not exist"
Nov 02 03:33:20.414008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.413935    4407 scope.go:122] "RemoveContainer" containerID="467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528"
Nov 02 03:33:20.423014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.422996    4407 scope.go:122] "RemoveContainer" containerID="467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528"
Nov 02 03:33:20.423282 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:33:20.423265    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528\": container with ID starting with 467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528 not found: ID does not exist" containerID="467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528"
Nov 02 03:33:20.423329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.423293    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528"} err="failed to get container status \"467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528\": rpc error: code = NotFound desc = could not find container \"467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528\": container with ID starting with 467b6f684c605d8891bda8f02e847fcd7521df58b2bf1de1f430f9645ea87528 not found: ID does not exist"
Nov 02 03:33:20.536799 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.536776    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:20.567139 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.567107    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"]
Nov 02 03:33:20.617842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.617759    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-gnf96"]
Nov 02 03:33:20.750691 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.750661    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"]
Nov 02 03:33:20.758553 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.758528    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-q9rnh"]
Nov 02 03:33:20.802239 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.802212    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:20.982839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:20.973311    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:22.351479 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:22.351447    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" path="/var/lib/kubelet/pods/14b6aabd-8241-47d5-93f2-494bfcef0106/volumes"
Nov 02 03:33:22.351914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:22.351899    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" path="/var/lib/kubelet/pods/300055ac-05f3-4374-ab56-6d62f61699b5/volumes"
Nov 02 03:33:26.204642 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:26.204558    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:31.073033 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:31.073004    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:33.135107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:33.135078    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"]
Nov 02 03:33:33.244771 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:33.244737    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"]
Nov 02 03:33:33.509779 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:33.509745    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:33.669670 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:33.669636    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:35.188812 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:33:35.188774    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:33:36.165693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:36.165615    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:38.136275 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.136213    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px" podUID="0df298c8-9673-4903-8be0-d11cf1380dd1" containerName="payload-processing" containerID="cri-o://cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc" gracePeriod=25
Nov 02 03:33:38.245715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.245680    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh" podUID="e4ead691-2863-43f4-b7d5-b723183a54bf" containerName="payload-pre-processing" containerID="cri-o://4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c" gracePeriod=25
Nov 02 03:33:38.412328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.412302    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:33:38.459789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.459754    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0df298c8-9673-4903-8be0-d11cf1380dd1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0df298c8-9673-4903-8be0-d11cf1380dd1-plugins-config-volume\") pod \"0df298c8-9673-4903-8be0-d11cf1380dd1\" (UID: \"0df298c8-9673-4903-8be0-d11cf1380dd1\") "
Nov 02 03:33:38.459981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.459825    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0df298c8-9673-4903-8be0-d11cf1380dd1-kube-api-access-xk8vt\" (UniqueName: \"kubernetes.io/projected/0df298c8-9673-4903-8be0-d11cf1380dd1-kube-api-access-xk8vt\") pod \"0df298c8-9673-4903-8be0-d11cf1380dd1\" (UID: \"0df298c8-9673-4903-8be0-d11cf1380dd1\") "
Nov 02 03:33:38.460182 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.460155    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0df298c8-9673-4903-8be0-d11cf1380dd1-plugins-config-volume" pod "0df298c8-9673-4903-8be0-d11cf1380dd1" (UID: "0df298c8-9673-4903-8be0-d11cf1380dd1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:33:38.460403 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.460380    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0df298c8-9673-4903-8be0-d11cf1380dd1-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:38.462363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.462336    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0df298c8-9673-4903-8be0-d11cf1380dd1-kube-api-access-xk8vt" pod "0df298c8-9673-4903-8be0-d11cf1380dd1" (UID: "0df298c8-9673-4903-8be0-d11cf1380dd1"). InnerVolumeSpecName "kube-api-access-xk8vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:33:38.509829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.509802    4407 generic.go:356] "Generic (PLEG): container finished" podID="0df298c8-9673-4903-8be0-d11cf1380dd1" containerID="cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc" exitCode=0
Nov 02 03:33:38.509963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.509837    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px" event={"ID":"0df298c8-9673-4903-8be0-d11cf1380dd1","Type":"ContainerDied","Data":"cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc"}
Nov 02 03:33:38.509963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.509891    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px" event={"ID":"0df298c8-9673-4903-8be0-d11cf1380dd1","Type":"ContainerDied","Data":"8a5428d98843b0fd179a681e53a155122b5ae4390926fbb5ebb9d07ee208ffbc"}
Nov 02 03:33:38.509963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.509911    4407 scope.go:122] "RemoveContainer" containerID="cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc"
Nov 02 03:33:38.509963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.509847    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"
Nov 02 03:33:38.510975 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.510834    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:33:38.511422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.511400    4407 generic.go:356] "Generic (PLEG): container finished" podID="e4ead691-2863-43f4-b7d5-b723183a54bf" containerID="4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c" exitCode=0
Nov 02 03:33:38.511527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.511436    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh" event={"ID":"e4ead691-2863-43f4-b7d5-b723183a54bf","Type":"ContainerDied","Data":"4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c"}
Nov 02 03:33:38.511527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.511474    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh" event={"ID":"e4ead691-2863-43f4-b7d5-b723183a54bf","Type":"ContainerDied","Data":"8236bb60659ffc24a987507ca63816ecec19225a2cc2240d65c2762bf8e365c4"}
Nov 02 03:33:38.522445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.522428    4407 scope.go:122] "RemoveContainer" containerID="cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc"
Nov 02 03:33:38.522745 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:33:38.522727    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc\": container with ID starting with cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc not found: ID does not exist" containerID="cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc"
Nov 02 03:33:38.522819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.522754    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc"} err="failed to get container status \"cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc\": rpc error: code = NotFound desc = could not find container \"cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc\": container with ID starting with cf8c943d1a064a361652b3f7a7fd64516948f1e9a1b440ec94689d04310263fc not found: ID does not exist"
Nov 02 03:33:38.522819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.522771    4407 scope.go:122] "RemoveContainer" containerID="4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c"
Nov 02 03:33:38.533401 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.533377    4407 scope.go:122] "RemoveContainer" containerID="4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c"
Nov 02 03:33:38.533734 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:33:38.533711    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c\": container with ID starting with 4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c not found: ID does not exist" containerID="4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c"
Nov 02 03:33:38.533820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.533746    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c"} err="failed to get container status \"4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c\": rpc error: code = NotFound desc = could not find container \"4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c\": container with ID starting with 4876b1e47fec5149e6bbef4abdc749b3aa5941e617da63b9b02a84ec7827e64c not found: ID does not exist"
Nov 02 03:33:38.560987 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.560953    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4ead691-2863-43f4-b7d5-b723183a54bf-kube-api-access-x5qm2\" (UniqueName: \"kubernetes.io/projected/e4ead691-2863-43f4-b7d5-b723183a54bf-kube-api-access-x5qm2\") pod \"e4ead691-2863-43f4-b7d5-b723183a54bf\" (UID: \"e4ead691-2863-43f4-b7d5-b723183a54bf\") "
Nov 02 03:33:38.561131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.561112    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4ead691-2863-43f4-b7d5-b723183a54bf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ead691-2863-43f4-b7d5-b723183a54bf-plugins-config-volume\") pod \"e4ead691-2863-43f4-b7d5-b723183a54bf\" (UID: \"e4ead691-2863-43f4-b7d5-b723183a54bf\") "
Nov 02 03:33:38.561482 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.561457    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ead691-2863-43f4-b7d5-b723183a54bf-plugins-config-volume" pod "e4ead691-2863-43f4-b7d5-b723183a54bf" (UID: "e4ead691-2863-43f4-b7d5-b723183a54bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:33:38.561687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.561670    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk8vt\" (UniqueName: \"kubernetes.io/projected/0df298c8-9673-4903-8be0-d11cf1380dd1-kube-api-access-xk8vt\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:38.561765 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.561688    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ead691-2863-43f4-b7d5-b723183a54bf-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:38.563233 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.563212    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ead691-2863-43f4-b7d5-b723183a54bf-kube-api-access-x5qm2" pod "e4ead691-2863-43f4-b7d5-b723183a54bf" (UID: "e4ead691-2863-43f4-b7d5-b723183a54bf"). InnerVolumeSpecName "kube-api-access-x5qm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:33:38.639931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.639877    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"]
Nov 02 03:33:38.643328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.643275    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f1a692-769dbb76cf-nm9px"]
Nov 02 03:33:38.663416 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:38.663396    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5qm2\" (UniqueName: \"kubernetes.io/projected/e4ead691-2863-43f4-b7d5-b723183a54bf-kube-api-access-x5qm2\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:33:39.305052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:39.305022    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:39.518573 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:39.518539    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"
Nov 02 03:33:39.689750 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:39.689714    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"]
Nov 02 03:33:39.693154 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:39.693129    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1a692-5779dfbc9-hx2bh"]
Nov 02 03:33:39.707788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:39.707762    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:40.222274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:40.222243    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:40.351459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:40.351430    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0df298c8-9673-4903-8be0-d11cf1380dd1" path="/var/lib/kubelet/pods/0df298c8-9673-4903-8be0-d11cf1380dd1/volumes"
Nov 02 03:33:40.351891 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:40.351876    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4ead691-2863-43f4-b7d5-b723183a54bf" path="/var/lib/kubelet/pods/e4ead691-2863-43f4-b7d5-b723183a54bf/volumes"
Nov 02 03:33:42.075538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.075508    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"]
Nov 02 03:33:42.076096 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076082    4407 cpu_manager.go:414] "containerMap: removing container" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy"
Nov 02 03:33:42.076141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076097    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy"
Nov 02 03:33:42.076141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076117    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0df298c8-9673-4903-8be0-d11cf1380dd1" containerName="payload-processing"
Nov 02 03:33:42.076141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076122    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0df298c8-9673-4903-8be0-d11cf1380dd1" containerName="payload-processing"
Nov 02 03:33:42.076141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076134    4407 cpu_manager.go:414] "containerMap: removing container" podUID="e4ead691-2863-43f4-b7d5-b723183a54bf" containerName="payload-pre-processing"
Nov 02 03:33:42.076141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076140    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4ead691-2863-43f4-b7d5-b723183a54bf" containerName="payload-pre-processing"
Nov 02 03:33:42.076295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076154    4407 cpu_manager.go:414] "containerMap: removing container" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy"
Nov 02 03:33:42.076295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076159    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy"
Nov 02 03:33:42.076295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076237    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="14b6aabd-8241-47d5-93f2-494bfcef0106" containerName="istio-proxy"
Nov 02 03:33:42.076295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076250    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="300055ac-05f3-4374-ab56-6d62f61699b5" containerName="istio-proxy"
Nov 02 03:33:42.076295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076258    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0df298c8-9673-4903-8be0-d11cf1380dd1" containerName="payload-processing"
Nov 02 03:33:42.076295 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.076264    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4ead691-2863-43f4-b7d5-b723183a54bf" containerName="payload-pre-processing"
Nov 02 03:33:42.081189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.081171    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:42.135296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.135261    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"]
Nov 02 03:33:42.182307 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.182272    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-018375-dockercfg-nslrm\""
Nov 02 03:33:42.182480 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.182378    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-018375\""
Nov 02 03:33:42.213930 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.213896    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ff28d1-a14f-4641-a567-a4aa90b65af5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v\" (UID: \"12ff28d1-a14f-4641-a567-a4aa90b65af5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:42.214092 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.214033    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ftf6\" (UniqueName: \"kubernetes.io/projected/12ff28d1-a14f-4641-a567-a4aa90b65af5-kube-api-access-4ftf6\") pod \"payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v\" (UID: \"12ff28d1-a14f-4641-a567-a4aa90b65af5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:42.315131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.315099    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ff28d1-a14f-4641-a567-a4aa90b65af5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v\" (UID: \"12ff28d1-a14f-4641-a567-a4aa90b65af5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:42.315307 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.315200    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ftf6\" (UniqueName: \"kubernetes.io/projected/12ff28d1-a14f-4641-a567-a4aa90b65af5-kube-api-access-4ftf6\") pod \"payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v\" (UID: \"12ff28d1-a14f-4641-a567-a4aa90b65af5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:42.315807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.315787    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ff28d1-a14f-4641-a567-a4aa90b65af5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v\" (UID: \"12ff28d1-a14f-4641-a567-a4aa90b65af5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:42.372761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.372689    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ftf6\" (UniqueName: \"kubernetes.io/projected/12ff28d1-a14f-4641-a567-a4aa90b65af5-kube-api-access-4ftf6\") pod \"payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v\" (UID: \"12ff28d1-a14f-4641-a567-a4aa90b65af5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:42.394341 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.394317    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:42.767764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.767738    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"]
Nov 02 03:33:42.768773 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:33:42.768747    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ff28d1_a14f_4641_a567_a4aa90b65af5.slice/crio-48034d149646a59b4341e2f08a58f32ac2a070df4f6e3de648d476786f164fcd WatchSource:0}: Error finding container 48034d149646a59b4341e2f08a58f32ac2a070df4f6e3de648d476786f164fcd: Status 404 returned error can't find the container with id 48034d149646a59b4341e2f08a58f32ac2a070df4f6e3de648d476786f164fcd
Nov 02 03:33:42.770795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.770778    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 03:33:42.981906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:42.981873    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:43.547243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:43.547205    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v" event={"ID":"12ff28d1-a14f-4641-a567-a4aa90b65af5","Type":"ContainerStarted","Data":"48034d149646a59b4341e2f08a58f32ac2a070df4f6e3de648d476786f164fcd"}
Nov 02 03:33:43.815530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:43.815443    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:48.579840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:48.579803    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v" event={"ID":"12ff28d1-a14f-4641-a567-a4aa90b65af5","Type":"ContainerStarted","Data":"3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d"}
Nov 02 03:33:48.580253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:48.579980    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:33:48.696486 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:48.696427    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v" podStartSLOduration=1.5495564609999999 podStartE2EDuration="6.696414183s" totalImagesPullingTime="5.146857722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:33:48.694701855 +0000 UTC m=+2772.490050420" watchObservedRunningTime="2026-11-02 03:33:48.696414183 +0000 UTC m=+2772.491762747"
Nov 02 03:33:49.473839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:49.473811    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:54.175747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:54.175711    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:54.190357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:54.190326    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:33:58.973974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:58.973940    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:33:59.588208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:33:59.588180    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:34:03.757874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:03.757845    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:05.189131 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:34:05.189098    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:34:05.566670 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:05.566617    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"]
Nov 02 03:34:05.712545 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:05.712481    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:05.871794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:05.871697    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:06.032873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:06.032838    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:08.224935 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:08.224884    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:08.791965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:08.791936    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:08.941522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:08.941481    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:09.428354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:09.428321    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"]
Nov 02 03:34:09.428910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:09.428541    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy" containerID="cri-o://2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68" gracePeriod=30
Nov 02 03:34:09.962329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:09.962300    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:10.190683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.190648    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:10.344913 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.344880    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:10.567395 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.567356    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt" podUID="c714449c-4518-43a2-b7dd-cdd4a6b64791" containerName="payload-pre-processing" containerID="cri-o://203df744412fed47128636df1c72dfddc97d194485a73cc31cf132727d54ffff" gracePeriod=25
Nov 02 03:34:10.710794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.710759    4407 generic.go:356] "Generic (PLEG): container finished" podID="c714449c-4518-43a2-b7dd-cdd4a6b64791" containerID="203df744412fed47128636df1c72dfddc97d194485a73cc31cf132727d54ffff" exitCode=0
Nov 02 03:34:10.710954 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.710841    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt" event={"ID":"c714449c-4518-43a2-b7dd-cdd4a6b64791","Type":"ContainerDied","Data":"203df744412fed47128636df1c72dfddc97d194485a73cc31cf132727d54ffff"}
Nov 02 03:34:10.822661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.822613    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:10.830575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.830555    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:34:10.899047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.899018    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c714449c-4518-43a2-b7dd-cdd4a6b64791-kube-api-access-vvlw6\" (UniqueName: \"kubernetes.io/projected/c714449c-4518-43a2-b7dd-cdd4a6b64791-kube-api-access-vvlw6\") pod \"c714449c-4518-43a2-b7dd-cdd4a6b64791\" (UID: \"c714449c-4518-43a2-b7dd-cdd4a6b64791\") "
Nov 02 03:34:10.899255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.899076    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c714449c-4518-43a2-b7dd-cdd4a6b64791-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c714449c-4518-43a2-b7dd-cdd4a6b64791-plugins-config-volume\") pod \"c714449c-4518-43a2-b7dd-cdd4a6b64791\" (UID: \"c714449c-4518-43a2-b7dd-cdd4a6b64791\") "
Nov 02 03:34:10.899475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.899454    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c714449c-4518-43a2-b7dd-cdd4a6b64791-plugins-config-volume" pod "c714449c-4518-43a2-b7dd-cdd4a6b64791" (UID: "c714449c-4518-43a2-b7dd-cdd4a6b64791"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:34:10.899657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.899637    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c714449c-4518-43a2-b7dd-cdd4a6b64791-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:10.901365 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:10.901338    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c714449c-4518-43a2-b7dd-cdd4a6b64791-kube-api-access-vvlw6" pod "c714449c-4518-43a2-b7dd-cdd4a6b64791" (UID: "c714449c-4518-43a2-b7dd-cdd4a6b64791"). InnerVolumeSpecName "kube-api-access-vvlw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:34:11.000835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:11.000809    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvlw6\" (UniqueName: \"kubernetes.io/projected/c714449c-4518-43a2-b7dd-cdd4a6b64791-kube-api-access-vvlw6\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:11.717680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:11.717654    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"
Nov 02 03:34:11.718199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:11.717649    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt" event={"ID":"c714449c-4518-43a2-b7dd-cdd4a6b64791","Type":"ContainerDied","Data":"9f894110477f12b022594d8cefdb7ab015911dc412705ff1b98bc8a0d35337f0"}
Nov 02 03:34:11.718199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:11.717778    4407 scope.go:122] "RemoveContainer" containerID="203df744412fed47128636df1c72dfddc97d194485a73cc31cf132727d54ffff"
Nov 02 03:34:11.885898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:11.885850    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"]
Nov 02 03:34:11.934974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:11.934945    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-69499c4ff9-drtjt"]
Nov 02 03:34:12.351515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:12.351482    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c714449c-4518-43a2-b7dd-cdd4a6b64791" path="/var/lib/kubelet/pods/c714449c-4518-43a2-b7dd-cdd4a6b64791/volumes"
Nov 02 03:34:14.699513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.699457    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:34:14.737574 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.737548    4407 generic.go:356] "Generic (PLEG): container finished" podID="a7b40fbd-084e-4593-aead-7c17531fb343" containerID="2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68" exitCode=0
Nov 02 03:34:14.737724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.737601    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"
Nov 02 03:34:14.737724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.737647    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" event={"ID":"a7b40fbd-084e-4593-aead-7c17531fb343","Type":"ContainerDied","Data":"2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68"}
Nov 02 03:34:14.737724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.737711    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r" event={"ID":"a7b40fbd-084e-4593-aead-7c17531fb343","Type":"ContainerDied","Data":"966479eddb5616a9cbdaa9cdeaa04e92a69f6757c9c1c68ec82abd67b8aa72af"}
Nov 02 03:34:14.737849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.737742    4407 scope.go:122] "RemoveContainer" containerID="2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68"
Nov 02 03:34:14.747406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.747390    4407 scope.go:122] "RemoveContainer" containerID="2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68"
Nov 02 03:34:14.747686 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:34:14.747667    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68\": container with ID starting with 2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68 not found: ID does not exist" containerID="2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68"
Nov 02 03:34:14.747739 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.747696    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68"} err="failed to get container status \"2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68\": rpc error: code = NotFound desc = could not find container \"2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68\": container with ID starting with 2e3d1d850aab64de885e9893f16f4bda76074fa5762b9f9b1d968902d220ff68 not found: ID does not exist"
Nov 02 03:34:14.851564 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.851532    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-data\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.851564 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.851574    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7b40fbd-084e-4593-aead-7c17531fb343-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b40fbd-084e-4593-aead-7c17531fb343-istio-podinfo\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.851856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.851655    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-kube-api-access-xdwvn\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-kube-api-access-xdwvn\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.851856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.851680    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7b40fbd-084e-4593-aead-7c17531fb343-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b40fbd-084e-4593-aead-7c17531fb343-istiod-ca-cert\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.851856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.851805    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-envoy\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.852022 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.851859    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-data" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:14.852022 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.851874    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-credential-socket\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.852022 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.852016    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-certs\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.852187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.852083    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-socket\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.852187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.852094    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b40fbd-084e-4593-aead-7c17531fb343-istiod-ca-cert" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:34:14.852187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.852136    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-credential-socket" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:14.852333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.852270    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-certs" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:14.852333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.852288    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-socket" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:14.852333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.852310    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-istio-token\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-istio-token\") pod \"a7b40fbd-084e-4593-aead-7c17531fb343\" (UID: \"a7b40fbd-084e-4593-aead-7c17531fb343\") "
Nov 02 03:34:14.853070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.853048    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:14.853179 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.853073    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b40fbd-084e-4593-aead-7c17531fb343-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:14.853179 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.853088    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:14.853179 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.853105    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:14.853179 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.853119    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:14.854172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.854145    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7b40fbd-084e-4593-aead-7c17531fb343-istio-podinfo" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:34:14.854271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.854180    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-envoy" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:14.854271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.854240    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-kube-api-access-xdwvn" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "kube-api-access-xdwvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:34:14.854510 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.854493    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-istio-token" pod "a7b40fbd-084e-4593-aead-7c17531fb343" (UID: "a7b40fbd-084e-4593-aead-7c17531fb343"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:34:14.954461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.954387    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b40fbd-084e-4593-aead-7c17531fb343-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:14.954461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.954416    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:14.954461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.954426    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b40fbd-084e-4593-aead-7c17531fb343-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:14.954461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:14.954436    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdwvn\" (UniqueName: \"kubernetes.io/projected/a7b40fbd-084e-4593-aead-7c17531fb343-kube-api-access-xdwvn\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:15.160094 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:15.160059    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"]
Nov 02 03:34:15.163983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:15.163950    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-fmg4r"]
Nov 02 03:34:15.346044 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:15.346009    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:16.351582 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:16.351546    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" path="/var/lib/kubelet/pods/a7b40fbd-084e-4593-aead-7c17531fb343/volumes"
Nov 02 03:34:16.822248 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:16.822217    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:16.984458 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:16.984422    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:17.112055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:17.111982    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"]
Nov 02 03:34:17.112212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:17.112173    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy" containerID="cri-o://75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7" gracePeriod=30
Nov 02 03:34:17.230649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:17.221268    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:20.678806 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:20.678771    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:22.385553 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.385479    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:34:22.448612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448584    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-data\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.448789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448615    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-socket\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.448789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448716    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-token\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-token\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.448789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448758    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-kube-api-access-8q29w\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-kube-api-access-8q29w\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.448947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448817    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-certs\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.448947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448889    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-data" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:22.448947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448901    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8ea9264-44db-4610-ba04-95a8ba22a66a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8ea9264-44db-4610-ba04-95a8ba22a66a-istiod-ca-cert\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.448947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448932    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-socket" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:22.449159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.448987    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-podinfo\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.449159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449030    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-credential-socket\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.449159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449049    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-envoy\") pod \"a8ea9264-44db-4610-ba04-95a8ba22a66a\" (UID: \"a8ea9264-44db-4610-ba04-95a8ba22a66a\") "
Nov 02 03:34:22.449159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449070    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-certs" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:22.449359 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449219    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8ea9264-44db-4610-ba04-95a8ba22a66a-istiod-ca-cert" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:34:22.449590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449568    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-credential-socket" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:22.449951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449924    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.450067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449954    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8ea9264-44db-4610-ba04-95a8ba22a66a-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.450067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449973    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.450067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.449988    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.450067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.450004    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.451354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.451329    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-podinfo" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:34:22.451447 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.451337    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-kube-api-access-8q29w" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "kube-api-access-8q29w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:34:22.451758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.451729    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-envoy" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:34:22.452018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.452000    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-token" pod "a8ea9264-44db-4610-ba04-95a8ba22a66a" (UID: "a8ea9264-44db-4610-ba04-95a8ba22a66a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:34:22.551064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.551034    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.551064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.551059    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.551064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.551069    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.551293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.551078    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q29w\" (UniqueName: \"kubernetes.io/projected/a8ea9264-44db-4610-ba04-95a8ba22a66a-kube-api-access-8q29w\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:34:22.790995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.790959    4407 generic.go:356] "Generic (PLEG): container finished" podID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerID="75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7" exitCode=0
Nov 02 03:34:22.791172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.791012    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"
Nov 02 03:34:22.791172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.791046    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" event={"ID":"a8ea9264-44db-4610-ba04-95a8ba22a66a","Type":"ContainerDied","Data":"75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7"}
Nov 02 03:34:22.791172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.791111    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx" event={"ID":"a8ea9264-44db-4610-ba04-95a8ba22a66a","Type":"ContainerDied","Data":"4adc8c9b79abe55af1a1df0e7e47b749747322d005b3f252445479528f261376"}
Nov 02 03:34:22.791172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.791136    4407 scope.go:122] "RemoveContainer" containerID="75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7"
Nov 02 03:34:22.801434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.801416    4407 scope.go:122] "RemoveContainer" containerID="9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a"
Nov 02 03:34:22.809984 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.809966    4407 scope.go:122] "RemoveContainer" containerID="75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7"
Nov 02 03:34:22.810244 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:34:22.810221    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7\": container with ID starting with 75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7 not found: ID does not exist" containerID="75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7"
Nov 02 03:34:22.810340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.810254    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7"} err="failed to get container status \"75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7\": rpc error: code = NotFound desc = could not find container \"75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7\": container with ID starting with 75e6f56eac42b9e9e9ca1218353d06e1109406bc40a3b17c71121f3c7a0d55e7 not found: ID does not exist"
Nov 02 03:34:22.810340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.810272    4407 scope.go:122] "RemoveContainer" containerID="9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a"
Nov 02 03:34:22.810520 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:34:22.810506    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a\": container with ID starting with 9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a not found: ID does not exist" containerID="9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a"
Nov 02 03:34:22.810566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.810524    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a"} err="failed to get container status \"9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a\": rpc error: code = NotFound desc = could not find container \"9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a\": container with ID starting with 9f23e7d0695c36bb047a4ddff4a285d9bd49e3c022ae8a3a78f039728674ec5a not found: ID does not exist"
Nov 02 03:34:22.958013 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.957981    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"]
Nov 02 03:34:22.962454 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:22.962426    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-045653-openshift-default-7dfd4bcd69-gjvsx"]
Nov 02 03:34:24.351456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:24.351423    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" path="/var/lib/kubelet/pods/a8ea9264-44db-4610-ba04-95a8ba22a66a/volumes"
Nov 02 03:34:24.482635 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:24.482592    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:25.108110 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:25.108082    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:27.875531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:27.875498    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:31.362836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:31.362795    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:34:32.583245 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:32.583216    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:35.189819 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:34:35.189772    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:34:37.493500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:37.493464    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:43.960511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:43.960476    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:44.159396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:44.159369    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:48.869983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:48.869903    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:34:55.474882 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:34:55.474838    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:01.964709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:01.964681    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:05.189674 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:35:05.189602    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:35:06.729462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:06.729424    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:07.589697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:07.589660    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:08.378300 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:08.378271    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v_12ff28d1-a14f-4641-a567-a4aa90b65af5/payload-processing/0.log"
Nov 02 03:35:08.594520 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:08.594494    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v_12ff28d1-a14f-4641-a567-a4aa90b65af5/payload-processing/0.log"
Nov 02 03:35:12.070405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:12.070375    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:15.694851 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:15.694819    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:16.182846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:16.182815    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:16.258177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:16.258143    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:35:17.271910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.271875    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"]
Nov 02 03:35:17.272401 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272386    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy"
Nov 02 03:35:17.272401 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272400    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy"
Nov 02 03:35:17.272525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272413    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy"
Nov 02 03:35:17.272525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272418    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy"
Nov 02 03:35:17.272525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272448    4407 cpu_manager.go:414] "containerMap: removing container" podUID="c714449c-4518-43a2-b7dd-cdd4a6b64791" containerName="payload-pre-processing"
Nov 02 03:35:17.272525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272454    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c714449c-4518-43a2-b7dd-cdd4a6b64791" containerName="payload-pre-processing"
Nov 02 03:35:17.272696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272577    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="c714449c-4518-43a2-b7dd-cdd4a6b64791" containerName="payload-pre-processing"
Nov 02 03:35:17.272696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272591    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy"
Nov 02 03:35:17.272696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.272597    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b40fbd-084e-4593-aead-7c17531fb343" containerName="istio-proxy"
Nov 02 03:35:17.276115 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.276098    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:17.332530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.332487    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"]
Nov 02 03:35:17.346225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.346197    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxqxq\" (UniqueName: \"kubernetes.io/projected/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-kube-api-access-bxqxq\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv\" (UID: \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:17.346411 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.346336    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv\" (UID: \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:17.378901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.378871    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0809c8\""
Nov 02 03:35:17.379083 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.378919    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-0809c8-dockercfg-s5px5\""
Nov 02 03:35:17.447790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.447757    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxqxq\" (UniqueName: \"kubernetes.io/projected/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-kube-api-access-bxqxq\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv\" (UID: \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:17.447979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.447843    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv\" (UID: \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:17.448557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.448532    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv\" (UID: \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:17.505566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.505536    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxqxq\" (UniqueName: \"kubernetes.io/projected/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-kube-api-access-bxqxq\") pod \"payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv\" (UID: \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:17.590206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.590124    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:17.964730 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:17.964702    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"]
Nov 02 03:35:17.964868 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:35:17.964808    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9cac2e3_1ff2_4c5e_8b24_4537afb8dd4d.slice/crio-a2b295bbbc40a0c10f26d33efef35d4e73311949b7a40d9b5785dd943910562d WatchSource:0}: Error finding container a2b295bbbc40a0c10f26d33efef35d4e73311949b7a40d9b5785dd943910562d: Status 404 returned error can't find the container with id a2b295bbbc40a0c10f26d33efef35d4e73311949b7a40d9b5785dd943910562d
Nov 02 03:35:18.061135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.061098    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"]
Nov 02 03:35:18.062031 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.062012    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy"
Nov 02 03:35:18.062108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.062036    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy"
Nov 02 03:35:18.062222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.062208    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8ea9264-44db-4610-ba04-95a8ba22a66a" containerName="istio-proxy"
Nov 02 03:35:18.066043 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.066024    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:18.118996 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.118962    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv" event={"ID":"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d","Type":"ContainerStarted","Data":"dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00"}
Nov 02 03:35:18.119177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.119022    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv" event={"ID":"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d","Type":"ContainerStarted","Data":"a2b295bbbc40a0c10f26d33efef35d4e73311949b7a40d9b5785dd943910562d"}
Nov 02 03:35:18.119177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.119054    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:18.120456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.120429    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"]
Nov 02 03:35:18.157877 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.157838    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzdkh\" (UniqueName: \"kubernetes.io/projected/13721185-dc40-47f5-99ac-e0d3be42cf49-kube-api-access-zzdkh\") pod \"payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk\" (UID: \"13721185-dc40-47f5-99ac-e0d3be42cf49\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:18.158041 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.157913    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13721185-dc40-47f5-99ac-e0d3be42cf49-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk\" (UID: \"13721185-dc40-47f5-99ac-e0d3be42cf49\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:18.259694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.259665    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzdkh\" (UniqueName: \"kubernetes.io/projected/13721185-dc40-47f5-99ac-e0d3be42cf49-kube-api-access-zzdkh\") pod \"payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk\" (UID: \"13721185-dc40-47f5-99ac-e0d3be42cf49\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:18.259877 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.259734    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13721185-dc40-47f5-99ac-e0d3be42cf49-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk\" (UID: \"13721185-dc40-47f5-99ac-e0d3be42cf49\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:18.260440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.260422    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13721185-dc40-47f5-99ac-e0d3be42cf49-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk\" (UID: \"13721185-dc40-47f5-99ac-e0d3be42cf49\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:18.302659 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.302571    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv" podStartSLOduration=1.302543709 podStartE2EDuration="1.302543709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:35:18.301729984 +0000 UTC m=+2862.097078550" watchObservedRunningTime="2026-11-02 03:35:18.302543709 +0000 UTC m=+2862.097892273"
Nov 02 03:35:18.316984 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.316958    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzdkh\" (UniqueName: \"kubernetes.io/projected/13721185-dc40-47f5-99ac-e0d3be42cf49-kube-api-access-zzdkh\") pod \"payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk\" (UID: \"13721185-dc40-47f5-99ac-e0d3be42cf49\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:18.380251 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.380222    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:18.775073 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:35:18.775039    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13721185_dc40_47f5_99ac_e0d3be42cf49.slice/crio-84c92b2655aa46cde40b3518aef8dc0b2ee1b225d6e437febcf0910d02268704 WatchSource:0}: Error finding container 84c92b2655aa46cde40b3518aef8dc0b2ee1b225d6e437febcf0910d02268704: Status 404 returned error can't find the container with id 84c92b2655aa46cde40b3518aef8dc0b2ee1b225d6e437febcf0910d02268704
Nov 02 03:35:18.775234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.775210    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"]
Nov 02 03:35:18.808899 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:18.808854    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:19.128931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:19.128836    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk" event={"ID":"13721185-dc40-47f5-99ac-e0d3be42cf49","Type":"ContainerStarted","Data":"d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624"}
Nov 02 03:35:19.128931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:19.128890    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:19.128931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:19.128906    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk" event={"ID":"13721185-dc40-47f5-99ac-e0d3be42cf49","Type":"ContainerStarted","Data":"84c92b2655aa46cde40b3518aef8dc0b2ee1b225d6e437febcf0910d02268704"}
Nov 02 03:35:19.246824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:19.246776    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk" podStartSLOduration=1.246761889 podStartE2EDuration="1.246761889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:35:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:35:19.246096485 +0000 UTC m=+2863.041445063" watchObservedRunningTime="2026-11-02 03:35:19.246761889 +0000 UTC m=+2863.042110454"
Nov 02 03:35:23.564731 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:23.564693    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:28.289887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:28.289855    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:29.131191 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:29.131155    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:35:30.136847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:30.136806    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:35:32.437023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:32.436993    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:32.961971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:32.961937    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:33.110239 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:33.110208    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:34.072568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:34.072530    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:34.305563 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:34.305528    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:34.506393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:34.506359    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:34.935860 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:34.935762    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:35.189641 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:35:35.189541    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:35:39.505333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:39.505298    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:40.615344 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:40.615308    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:40.779173 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:40.779134    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:40.943849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:40.943743    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:42.256797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:42.256768    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:42.850010 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:42.849974    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:43.052648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:43.052580    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:44.498817 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:44.498785    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:44.674532 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:44.674497    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:46.776476 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:46.776442    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:46.953417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:46.953378    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:47.165281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:47.165194    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:49.153198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:49.153167    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:49.983737 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:49.983701    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:50.165029 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:50.164981    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:50.879286 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:50.879253    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:51.156075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:51.155988    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:51.354092 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:51.354038    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:51.778159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:51.778124    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:55.197069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:55.197028    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:35:57.531788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.531755    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"]
Nov 02 03:35:57.532297 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.532047    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz" podUID="083082f5-1ba8-46ea-a8e7-e0d588a66a42" containerName="maas-api" containerID="cri-o://efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992" gracePeriod=30
Nov 02 03:35:57.667962 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.667912    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:57.832524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.832497    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:35:57.885177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.885127    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:57.889665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.889613    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls\") pod \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") "
Nov 02 03:35:57.889748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.889707    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-metrics-certs\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-metrics-certs\") pod \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") "
Nov 02 03:35:57.889809 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.889782    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/083082f5-1ba8-46ea-a8e7-e0d588a66a42-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/083082f5-1ba8-46ea-a8e7-e0d588a66a42-openshift-service-ca\") pod \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") "
Nov 02 03:35:57.889873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.889848    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/083082f5-1ba8-46ea-a8e7-e0d588a66a42-kube-api-access-p8t4f\" (UniqueName: \"kubernetes.io/projected/083082f5-1ba8-46ea-a8e7-e0d588a66a42-kube-api-access-p8t4f\") pod \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\" (UID: \"083082f5-1ba8-46ea-a8e7-e0d588a66a42\") "
Nov 02 03:35:57.890189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.890165    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/083082f5-1ba8-46ea-a8e7-e0d588a66a42-openshift-service-ca" pod "083082f5-1ba8-46ea-a8e7-e0d588a66a42" (UID: "083082f5-1ba8-46ea-a8e7-e0d588a66a42"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:35:57.890450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.890416    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/083082f5-1ba8-46ea-a8e7-e0d588a66a42-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:35:57.891848 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.891824    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-metrics-certs" pod "083082f5-1ba8-46ea-a8e7-e0d588a66a42" (UID: "083082f5-1ba8-46ea-a8e7-e0d588a66a42"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:35:57.892011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.891994    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls" pod "083082f5-1ba8-46ea-a8e7-e0d588a66a42" (UID: "083082f5-1ba8-46ea-a8e7-e0d588a66a42"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:35:57.892062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.892021    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/083082f5-1ba8-46ea-a8e7-e0d588a66a42-kube-api-access-p8t4f" pod "083082f5-1ba8-46ea-a8e7-e0d588a66a42" (UID: "083082f5-1ba8-46ea-a8e7-e0d588a66a42"). InnerVolumeSpecName "kube-api-access-p8t4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:35:57.991581 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.991496    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8t4f\" (UniqueName: \"kubernetes.io/projected/083082f5-1ba8-46ea-a8e7-e0d588a66a42-kube-api-access-p8t4f\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:35:57.991581 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.991533    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:35:57.991581 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:57.991548    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/083082f5-1ba8-46ea-a8e7-e0d588a66a42-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:35:58.044373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.044335    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:58.208677 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.208554    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:58.357569 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.357536    4407 generic.go:356] "Generic (PLEG): container finished" podID="083082f5-1ba8-46ea-a8e7-e0d588a66a42" containerID="efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992" exitCode=0
Nov 02 03:35:58.357767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.357574    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz" event={"ID":"083082f5-1ba8-46ea-a8e7-e0d588a66a42","Type":"ContainerDied","Data":"efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992"}
Nov 02 03:35:58.357767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.357590    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"
Nov 02 03:35:58.357767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.357650    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz" event={"ID":"083082f5-1ba8-46ea-a8e7-e0d588a66a42","Type":"ContainerDied","Data":"f7ca64450f24dac5997d985b5ccd52421f96ba6cf118ac485e0f306d3ef5be42"}
Nov 02 03:35:58.357767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.357683    4407 scope.go:122] "RemoveContainer" containerID="efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992"
Nov 02 03:35:58.364006 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.363967    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:58.370834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.370503    4407 scope.go:122] "RemoveContainer" containerID="efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992"
Nov 02 03:35:58.370952 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:35:58.370935    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992\": container with ID starting with efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992 not found: ID does not exist" containerID="efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992"
Nov 02 03:35:58.371012 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.370962    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992"} err="failed to get container status \"efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992\": rpc error: code = NotFound desc = could not find container \"efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992\": container with ID starting with efa5f255cf0986364a002bbda788f4337b1413d2af645dd5122fb38e5de6b992 not found: ID does not exist"
Nov 02 03:35:58.528350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.528312    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"]
Nov 02 03:35:58.532396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:58.532371    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-abc348-f6dc75f6b-4fskz"]
Nov 02 03:35:59.547304 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:59.547272    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:59.711018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:59.710984    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:35:59.876159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:35:59.876068    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:00.230191 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:00.230105    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:00.351594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:00.351564    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="083082f5-1ba8-46ea-a8e7-e0d588a66a42" path="/var/lib/kubelet/pods/083082f5-1ba8-46ea-a8e7-e0d588a66a42/volumes"
Nov 02 03:36:00.418207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:00.418179    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:00.576130 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:00.576098    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:04.964170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:04.964139    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:05.133141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:05.133107    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:05.207082 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:36:05.207043    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:36:06.354257 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:06.354223    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:06.501606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:06.501573    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:06.694030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:06.693953    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:06.851148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:06.851116    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:08.073592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:08.073558    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:08.242584 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:08.242554    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:10.573553 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:10.573525    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:17.462934 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:17.462898    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:22.581076 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:22.580997    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:25.490807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:25.490776    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:27.817839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:27.817791    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:32.759861 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:32.759829    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:35.189746 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:36:35.189709    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:36:37.503945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:37.503915    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:41.190151 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.190117    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:36:41.521379 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.521348    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"]
Nov 02 03:36:41.521588 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.521565    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6" podUID="924714d8-000e-4584-b58e-d9631ac77a20" containerName="maas-api" containerID="cri-o://af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671" gracePeriod=30
Nov 02 03:36:41.641614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.641583    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:41.802741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.802715    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:36:41.936870 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.936832    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-metrics-certs\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-metrics-certs\") pod \"924714d8-000e-4584-b58e-d9631ac77a20\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") "
Nov 02 03:36:41.937066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.936924    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/924714d8-000e-4584-b58e-d9631ac77a20-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/924714d8-000e-4584-b58e-d9631ac77a20-openshift-service-ca\") pod \"924714d8-000e-4584-b58e-d9631ac77a20\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") "
Nov 02 03:36:41.937066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.937056    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/924714d8-000e-4584-b58e-d9631ac77a20-kube-api-access-j6qrk\" (UniqueName: \"kubernetes.io/projected/924714d8-000e-4584-b58e-d9631ac77a20-kube-api-access-j6qrk\") pod \"924714d8-000e-4584-b58e-d9631ac77a20\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") "
Nov 02 03:36:41.937185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.937118    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls\") pod \"924714d8-000e-4584-b58e-d9631ac77a20\" (UID: \"924714d8-000e-4584-b58e-d9631ac77a20\") "
Nov 02 03:36:41.938433 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.938400    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/924714d8-000e-4584-b58e-d9631ac77a20-openshift-service-ca" pod "924714d8-000e-4584-b58e-d9631ac77a20" (UID: "924714d8-000e-4584-b58e-d9631ac77a20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:36:41.944128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.941158    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls" pod "924714d8-000e-4584-b58e-d9631ac77a20" (UID: "924714d8-000e-4584-b58e-d9631ac77a20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:36:41.949856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.944507    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/924714d8-000e-4584-b58e-d9631ac77a20-kube-api-access-j6qrk" pod "924714d8-000e-4584-b58e-d9631ac77a20" (UID: "924714d8-000e-4584-b58e-d9631ac77a20"). InnerVolumeSpecName "kube-api-access-j6qrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:36:41.949856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:41.944916    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-metrics-certs" pod "924714d8-000e-4584-b58e-d9631ac77a20" (UID: "924714d8-000e-4584-b58e-d9631ac77a20"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:36:42.039728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.039642    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:36:42.039728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.039673    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924714d8-000e-4584-b58e-d9631ac77a20-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:36:42.039728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.039684    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/924714d8-000e-4584-b58e-d9631ac77a20-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:36:42.039728 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.039695    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6qrk\" (UniqueName: \"kubernetes.io/projected/924714d8-000e-4584-b58e-d9631ac77a20-kube-api-access-j6qrk\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:36:42.171746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.171719    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:42.337479 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.337399    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:42.617258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.617164    4407 generic.go:356] "Generic (PLEG): container finished" podID="924714d8-000e-4584-b58e-d9631ac77a20" containerID="af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671" exitCode=0
Nov 02 03:36:42.617258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.617214    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"
Nov 02 03:36:42.617258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.617248    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6" event={"ID":"924714d8-000e-4584-b58e-d9631ac77a20","Type":"ContainerDied","Data":"af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671"}
Nov 02 03:36:42.617540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.617305    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6" event={"ID":"924714d8-000e-4584-b58e-d9631ac77a20","Type":"ContainerDied","Data":"592eba2b22d43afbca1daadfac8d4af0367c79ae5f0085faa35eb5e622feec68"}
Nov 02 03:36:42.617540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.617334    4407 scope.go:122] "RemoveContainer" containerID="af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671"
Nov 02 03:36:42.626964 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.626947    4407 scope.go:122] "RemoveContainer" containerID="af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671"
Nov 02 03:36:42.627221 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:36:42.627202    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671\": container with ID starting with af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671 not found: ID does not exist" containerID="af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671"
Nov 02 03:36:42.627287 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.627234    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671"} err="failed to get container status \"af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671\": rpc error: code = NotFound desc = could not find container \"af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671\": container with ID starting with af0edf350e23cc77796f7743b5333261b50f9089e3aee99fb9b634d731e0c671 not found: ID does not exist"
Nov 02 03:36:42.783699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.783670    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"]
Nov 02 03:36:42.834054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:42.834024    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd87f0-6b94576688-7t2l6"]
Nov 02 03:36:43.254859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:43.254824    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:43.481054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:43.481021    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:43.644010 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:43.643929    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:44.175542 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:44.175512    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:44.352183 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:44.352152    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="924714d8-000e-4584-b58e-d9631ac77a20" path="/var/lib/kubelet/pods/924714d8-000e-4584-b58e-d9631ac77a20/volumes"
Nov 02 03:36:44.838951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:44.838916    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:48.462249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:48.462218    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:48.617472 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:48.617444    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:48.776909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:48.776879    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:49.202968 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:49.202855    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:53.562895 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:53.562854    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:36:59.981991 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:36:59.981956    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:04.640180 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:04.640143    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:04.790160 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:04.790134    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:04.952786 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:04.952699    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:05.191377 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:05.191340    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:37:09.577507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:09.577461    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:13.650214 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:13.650174    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"]
Nov 02 03:37:13.650680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:13.650395    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7" podUID="aa21aeaa-e46e-4090-b212-1d0bc2f608db" containerName="maas-api" containerID="cri-o://df60963b175a3fa387e72a2dd58d98e10eba1b789225250490872a5721d59c07" gracePeriod=30
Nov 02 03:37:13.783468 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:13.783432    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:13.810611 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:13.810569    4407 generic.go:356] "Generic (PLEG): container finished" podID="aa21aeaa-e46e-4090-b212-1d0bc2f608db" containerID="df60963b175a3fa387e72a2dd58d98e10eba1b789225250490872a5721d59c07" exitCode=0
Nov 02 03:37:13.810800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:13.810617    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7" event={"ID":"aa21aeaa-e46e-4090-b212-1d0bc2f608db","Type":"ContainerDied","Data":"df60963b175a3fa387e72a2dd58d98e10eba1b789225250490872a5721d59c07"}
Nov 02 03:37:13.924988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:13.924964    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:37:14.022046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.022005    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls\") pod \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") "
Nov 02 03:37:14.022253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.022076    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa21aeaa-e46e-4090-b212-1d0bc2f608db-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa21aeaa-e46e-4090-b212-1d0bc2f608db-openshift-service-ca\") pod \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") "
Nov 02 03:37:14.022253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.022212    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa21aeaa-e46e-4090-b212-1d0bc2f608db-kube-api-access-9rl8m\" (UniqueName: \"kubernetes.io/projected/aa21aeaa-e46e-4090-b212-1d0bc2f608db-kube-api-access-9rl8m\") pod \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") "
Nov 02 03:37:14.022253 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.022249    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-metrics-certs\") pod \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\" (UID: \"aa21aeaa-e46e-4090-b212-1d0bc2f608db\") "
Nov 02 03:37:14.022485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.022457    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa21aeaa-e46e-4090-b212-1d0bc2f608db-openshift-service-ca" pod "aa21aeaa-e46e-4090-b212-1d0bc2f608db" (UID: "aa21aeaa-e46e-4090-b212-1d0bc2f608db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:37:14.022777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.022753    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa21aeaa-e46e-4090-b212-1d0bc2f608db-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:14.024313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.024288    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls" pod "aa21aeaa-e46e-4090-b212-1d0bc2f608db" (UID: "aa21aeaa-e46e-4090-b212-1d0bc2f608db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:37:14.024408 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.024385    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa21aeaa-e46e-4090-b212-1d0bc2f608db-kube-api-access-9rl8m" pod "aa21aeaa-e46e-4090-b212-1d0bc2f608db" (UID: "aa21aeaa-e46e-4090-b212-1d0bc2f608db"). InnerVolumeSpecName "kube-api-access-9rl8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:37:14.024459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.024444    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-metrics-certs" pod "aa21aeaa-e46e-4090-b212-1d0bc2f608db" (UID: "aa21aeaa-e46e-4090-b212-1d0bc2f608db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:37:14.119651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.119601    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"]
Nov 02 03:37:14.119879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.119858    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk" podUID="13721185-dc40-47f5-99ac-e0d3be42cf49" containerName="payload-processing" containerID="cri-o://d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624" gracePeriod=30
Nov 02 03:37:14.124341 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.124311    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:14.124341 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.124342    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rl8m\" (UniqueName: \"kubernetes.io/projected/aa21aeaa-e46e-4090-b212-1d0bc2f608db-kube-api-access-9rl8m\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:14.124489 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.124353    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa21aeaa-e46e-4090-b212-1d0bc2f608db-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:14.177091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.177005    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"]
Nov 02 03:37:14.177232 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.177212    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv" podUID="d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d" containerName="payload-pre-processing" containerID="cri-o://dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00" gracePeriod=30
Nov 02 03:37:14.294345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.294318    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:14.446568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.446538    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:14.448906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.448888    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:37:14.467563 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.467537    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:37:14.653653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.632771    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-kube-api-access-bxqxq\" (UniqueName: \"kubernetes.io/projected/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-kube-api-access-bxqxq\") pod \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\" (UID: \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\") "
Nov 02 03:37:14.653653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.632873    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13721185-dc40-47f5-99ac-e0d3be42cf49-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13721185-dc40-47f5-99ac-e0d3be42cf49-plugins-config-volume\") pod \"13721185-dc40-47f5-99ac-e0d3be42cf49\" (UID: \"13721185-dc40-47f5-99ac-e0d3be42cf49\") "
Nov 02 03:37:14.653653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.632976    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-plugins-config-volume\") pod \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\" (UID: \"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d\") "
Nov 02 03:37:14.653653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.633015    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13721185-dc40-47f5-99ac-e0d3be42cf49-kube-api-access-zzdkh\" (UniqueName: \"kubernetes.io/projected/13721185-dc40-47f5-99ac-e0d3be42cf49-kube-api-access-zzdkh\") pod \"13721185-dc40-47f5-99ac-e0d3be42cf49\" (UID: \"13721185-dc40-47f5-99ac-e0d3be42cf49\") "
Nov 02 03:37:14.653653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.636673    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-plugins-config-volume" pod "d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d" (UID: "d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:37:14.653653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.636865    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13721185-dc40-47f5-99ac-e0d3be42cf49-plugins-config-volume" pod "13721185-dc40-47f5-99ac-e0d3be42cf49" (UID: "13721185-dc40-47f5-99ac-e0d3be42cf49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:37:14.653653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.641220    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13721185-dc40-47f5-99ac-e0d3be42cf49-kube-api-access-zzdkh" pod "13721185-dc40-47f5-99ac-e0d3be42cf49" (UID: "13721185-dc40-47f5-99ac-e0d3be42cf49"). InnerVolumeSpecName "kube-api-access-zzdkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:37:14.673270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.673232    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-kube-api-access-bxqxq" pod "d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d" (UID: "d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d"). InnerVolumeSpecName "kube-api-access-bxqxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:37:14.734444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.734373    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13721185-dc40-47f5-99ac-e0d3be42cf49-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:14.734444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.734399    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:14.734444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.734411    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzdkh\" (UniqueName: \"kubernetes.io/projected/13721185-dc40-47f5-99ac-e0d3be42cf49-kube-api-access-zzdkh\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:14.734444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.734425    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxqxq\" (UniqueName: \"kubernetes.io/projected/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d-kube-api-access-bxqxq\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:14.817417 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.817384    4407 generic.go:356] "Generic (PLEG): container finished" podID="13721185-dc40-47f5-99ac-e0d3be42cf49" containerID="d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624" exitCode=0
Nov 02 03:37:14.817608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.817498    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"
Nov 02 03:37:14.817608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.817503    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk" event={"ID":"13721185-dc40-47f5-99ac-e0d3be42cf49","Type":"ContainerDied","Data":"d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624"}
Nov 02 03:37:14.817746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.817645    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk" event={"ID":"13721185-dc40-47f5-99ac-e0d3be42cf49","Type":"ContainerDied","Data":"84c92b2655aa46cde40b3518aef8dc0b2ee1b225d6e437febcf0910d02268704"}
Nov 02 03:37:14.817746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.817685    4407 scope.go:122] "RemoveContainer" containerID="d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624"
Nov 02 03:37:14.819036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.819010    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"
Nov 02 03:37:14.819109 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.819076    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7" event={"ID":"aa21aeaa-e46e-4090-b212-1d0bc2f608db","Type":"ContainerDied","Data":"4cea1248735746c87ae3ca50f82d1ce78c1c4de4a8782dea6a1dc6ecaa2faa57"}
Nov 02 03:37:14.824546 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.824378    4407 generic.go:356] "Generic (PLEG): container finished" podID="d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d" containerID="dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00" exitCode=0
Nov 02 03:37:14.824546 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.824421    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv" event={"ID":"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d","Type":"ContainerDied","Data":"dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00"}
Nov 02 03:37:14.824546 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.824458    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv" event={"ID":"d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d","Type":"ContainerDied","Data":"a2b295bbbc40a0c10f26d33efef35d4e73311949b7a40d9b5785dd943910562d"}
Nov 02 03:37:14.824776 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.824608    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"
Nov 02 03:37:14.834470 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.834454    4407 scope.go:122] "RemoveContainer" containerID="d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624"
Nov 02 03:37:14.834823 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:14.834798    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624\": container with ID starting with d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624 not found: ID does not exist" containerID="d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624"
Nov 02 03:37:14.834911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.834832    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624"} err="failed to get container status \"d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624\": rpc error: code = NotFound desc = could not find container \"d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624\": container with ID starting with d18b5b5ef447555ab226eb0ef28eac15ba442252241f60fad866a9f8d4cc5624 not found: ID does not exist"
Nov 02 03:37:14.834911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.834855    4407 scope.go:122] "RemoveContainer" containerID="df60963b175a3fa387e72a2dd58d98e10eba1b789225250490872a5721d59c07"
Nov 02 03:37:14.843977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.843954    4407 scope.go:122] "RemoveContainer" containerID="dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00"
Nov 02 03:37:14.852063 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.852031    4407 scope.go:122] "RemoveContainer" containerID="dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00"
Nov 02 03:37:14.852343 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:14.852304    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00\": container with ID starting with dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00 not found: ID does not exist" containerID="dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00"
Nov 02 03:37:14.852425 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.852351    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00"} err="failed to get container status \"dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00\": rpc error: code = NotFound desc = could not find container \"dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00\": container with ID starting with dabefa17191014ef9579415b7403c04f69a0664650f6043eeabd0348b09a1d00 not found: ID does not exist"
Nov 02 03:37:14.985574 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.985480    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"]
Nov 02 03:37:14.988631 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:14.988602    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0809c8-66c9f5dbc9-8k2sv"]
Nov 02 03:37:15.108211 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:15.108175    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"]
Nov 02 03:37:15.110506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:15.110486    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0809c8-5fb6d4f664-fbphk"]
Nov 02 03:37:15.219715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:15.219682    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"]
Nov 02 03:37:15.221916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:15.221898    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0809c8-645cd56848-j2rb7"]
Nov 02 03:37:15.353112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:15.353078    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:15.573223 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:15.573167    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:15.738426 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:15.738355    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:16.221359 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:16.221311    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:16.350859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:16.350827    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13721185-dc40-47f5-99ac-e0d3be42cf49" path="/var/lib/kubelet/pods/13721185-dc40-47f5-99ac-e0d3be42cf49/volumes"
Nov 02 03:37:16.351270 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:16.351257    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa21aeaa-e46e-4090-b212-1d0bc2f608db" path="/var/lib/kubelet/pods/aa21aeaa-e46e-4090-b212-1d0bc2f608db/volumes"
Nov 02 03:37:16.351604 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:16.351592    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d" path="/var/lib/kubelet/pods/d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d/volumes"
Nov 02 03:37:18.044317 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:18.044277    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:37:20.770838 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:20.770800    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:22.369135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:22.369102    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:22.529566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:22.529518    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:22.701207 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:22.701126    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:24.104513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:24.104479    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:24.675827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:24.675793    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:25.173860 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:25.173816    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:25.532202 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:25.532164    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:26.134676 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:26.134650    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:26.690797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:26.690763    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:28.407396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:28.407358    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:28.579682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:28.579652    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:31.211816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:31.211779    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:32.203218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:32.203180    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:32.369212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:32.369174    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:32.494784 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:32.494759    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"]
Nov 02 03:37:32.494973 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:32.494953    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy" containerID="cri-o://a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22" gracePeriod=30
Nov 02 03:37:32.535669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:32.535641    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:32.721654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:32.712971    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:35.189980 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:35.189937    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:37:36.151402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:36.151369    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:37.301405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.301374    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:37.460832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.460802    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:37.756175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.756151    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:37:37.827260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827230    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istiod-ca-cert\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.827421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827307    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-kube-api-access-v8w8l\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-kube-api-access-v8w8l\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.827495 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827472    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-data\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.827667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827639    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-socket\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.827667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827656    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istiod-ca-cert" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:37:37.827844 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827731    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-data" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:37:37.827844 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827697    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-envoy\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.827844 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827814    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-socket" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:37:37.827844 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827829    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-token\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-token\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.828059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827856    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-podinfo\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.828059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827925    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-certs\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.828059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.827948    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-credential-socket\") pod \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\" (UID: \"06c937a7-e688-49e8-ab6e-6726e3afa7a9\") "
Nov 02 03:37:37.828230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.828153    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-certs" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:37:37.828430 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.828411    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.828534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.828463    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.828534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.828478    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.828534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.828483    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-credential-socket" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:37:37.828534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.828493    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.829771 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.829746    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-kube-api-access-v8w8l" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "kube-api-access-v8w8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:37:37.830002 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.829984    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-token" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:37:37.830300 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.830285    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-envoy" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:37:37.830465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.830452    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-podinfo" pod "06c937a7-e688-49e8-ab6e-6726e3afa7a9" (UID: "06c937a7-e688-49e8-ab6e-6726e3afa7a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:37:37.929596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.929516    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8w8l\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-kube-api-access-v8w8l\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.929596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.929542    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.929596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.929551    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.929596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.929561    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06c937a7-e688-49e8-ab6e-6726e3afa7a9-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.929596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.929571    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06c937a7-e688-49e8-ab6e-6726e3afa7a9-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:37.975243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.975205    4407 generic.go:356] "Generic (PLEG): container finished" podID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerID="a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22" exitCode=0
Nov 02 03:37:37.975407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.975264    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"
Nov 02 03:37:37.975407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.975280    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" event={"ID":"06c937a7-e688-49e8-ab6e-6726e3afa7a9","Type":"ContainerDied","Data":"a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22"}
Nov 02 03:37:37.975407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.975323    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8" event={"ID":"06c937a7-e688-49e8-ab6e-6726e3afa7a9","Type":"ContainerDied","Data":"728a6677d9e314887bbe22d7b03ebec0833a8117ccc43953ea66095adafbcc5d"}
Nov 02 03:37:37.975407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.975345    4407 scope.go:122] "RemoveContainer" containerID="a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22"
Nov 02 03:37:37.985527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.985506    4407 scope.go:122] "RemoveContainer" containerID="09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701"
Nov 02 03:37:37.994321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.994303    4407 scope.go:122] "RemoveContainer" containerID="a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22"
Nov 02 03:37:37.994597 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:37.994578    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22\": container with ID starting with a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22 not found: ID does not exist" containerID="a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22"
Nov 02 03:37:37.994651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.994607    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22"} err="failed to get container status \"a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22\": rpc error: code = NotFound desc = could not find container \"a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22\": container with ID starting with a5f837c63aef22d07b551e057a813b1f450c76f7faf04bfa953170b3a302af22 not found: ID does not exist"
Nov 02 03:37:37.994651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.994640    4407 scope.go:122] "RemoveContainer" containerID="09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701"
Nov 02 03:37:37.994895 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:37.994878    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701\": container with ID starting with 09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701 not found: ID does not exist" containerID="09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701"
Nov 02 03:37:37.994948 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:37.994901    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701"} err="failed to get container status \"09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701\": rpc error: code = NotFound desc = could not find container \"09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701\": container with ID starting with 09164688ea921d00be3dba00a396403d1fff087975c98d36fef51e96bf0d8701 not found: ID does not exist"
Nov 02 03:37:38.142180 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:38.142144    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"]
Nov 02 03:37:38.145407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:38.145377    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-dd87f0-openshift-default-566d797d85-fqvt8"]
Nov 02 03:37:38.351886 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:38.351856    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" path="/var/lib/kubelet/pods/06c937a7-e688-49e8-ab6e-6726e3afa7a9/volumes"
Nov 02 03:37:41.267853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:41.267823    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:43.993158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:43.993129    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"]
Nov 02 03:37:43.993661 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:43.993351    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c" podUID="ecf9b24a-128d-4247-92c0-e8b70113e800" containerName="maas-api" containerID="cri-o://c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6" gracePeriod=30
Nov 02 03:37:44.254590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.254567    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:37:44.405951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.405916    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls\") pod \"ecf9b24a-128d-4247-92c0-e8b70113e800\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") "
Nov 02 03:37:44.406146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.406002    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ecf9b24a-128d-4247-92c0-e8b70113e800-kube-api-access-dbzwc\" (UniqueName: \"kubernetes.io/projected/ecf9b24a-128d-4247-92c0-e8b70113e800-kube-api-access-dbzwc\") pod \"ecf9b24a-128d-4247-92c0-e8b70113e800\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") "
Nov 02 03:37:44.406146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.406102    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ecf9b24a-128d-4247-92c0-e8b70113e800-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf9b24a-128d-4247-92c0-e8b70113e800-openshift-service-ca\") pod \"ecf9b24a-128d-4247-92c0-e8b70113e800\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") "
Nov 02 03:37:44.406307 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.406287    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-metrics-certs\") pod \"ecf9b24a-128d-4247-92c0-e8b70113e800\" (UID: \"ecf9b24a-128d-4247-92c0-e8b70113e800\") "
Nov 02 03:37:44.406503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.406474    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecf9b24a-128d-4247-92c0-e8b70113e800-openshift-service-ca" pod "ecf9b24a-128d-4247-92c0-e8b70113e800" (UID: "ecf9b24a-128d-4247-92c0-e8b70113e800"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:37:44.406895 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.406871    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf9b24a-128d-4247-92c0-e8b70113e800-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:44.408406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.408377    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf9b24a-128d-4247-92c0-e8b70113e800-kube-api-access-dbzwc" pod "ecf9b24a-128d-4247-92c0-e8b70113e800" (UID: "ecf9b24a-128d-4247-92c0-e8b70113e800"). InnerVolumeSpecName "kube-api-access-dbzwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:37:44.408521 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.408456    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls" pod "ecf9b24a-128d-4247-92c0-e8b70113e800" (UID: "ecf9b24a-128d-4247-92c0-e8b70113e800"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:37:44.409178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.409151    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-metrics-certs" pod "ecf9b24a-128d-4247-92c0-e8b70113e800" (UID: "ecf9b24a-128d-4247-92c0-e8b70113e800"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:37:44.508293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.508210    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:44.508293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.508237    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf9b24a-128d-4247-92c0-e8b70113e800-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:44.508293 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:44.508247    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbzwc\" (UniqueName: \"kubernetes.io/projected/ecf9b24a-128d-4247-92c0-e8b70113e800-kube-api-access-dbzwc\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:45.028940 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.028905    4407 generic.go:356] "Generic (PLEG): container finished" podID="ecf9b24a-128d-4247-92c0-e8b70113e800" containerID="c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6" exitCode=0
Nov 02 03:37:45.029424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.028972    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c" event={"ID":"ecf9b24a-128d-4247-92c0-e8b70113e800","Type":"ContainerDied","Data":"c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6"}
Nov 02 03:37:45.029424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.028978    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"
Nov 02 03:37:45.029424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.029037    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c" event={"ID":"ecf9b24a-128d-4247-92c0-e8b70113e800","Type":"ContainerDied","Data":"6d105e69439d3f36da75fc9909020dafc8ee9799958ec42a1e39f7d8d10379e9"}
Nov 02 03:37:45.029424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.029059    4407 scope.go:122] "RemoveContainer" containerID="c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6"
Nov 02 03:37:45.039882 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.039857    4407 scope.go:122] "RemoveContainer" containerID="c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6"
Nov 02 03:37:45.040168 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:45.040148    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6\": container with ID starting with c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6 not found: ID does not exist" containerID="c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6"
Nov 02 03:37:45.040218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.040178    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6"} err="failed to get container status \"c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6\": rpc error: code = NotFound desc = could not find container \"c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6\": container with ID starting with c829f79e949b5afffed797e8716fcee42d5f668cf5b56a9d48ab1b08779db0c6 not found: ID does not exist"
Nov 02 03:37:45.188741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.188704    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:45.195444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.195417    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"]
Nov 02 03:37:45.246397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:45.246367    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d94cc3-55f675b9d8-gcs9c"]
Nov 02 03:37:46.351255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:46.351222    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ecf9b24a-128d-4247-92c0-e8b70113e800" path="/var/lib/kubelet/pods/ecf9b24a-128d-4247-92c0-e8b70113e800/volumes"
Nov 02 03:37:46.466489 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:46.466454    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:47.790885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:47.790841    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:37:50.967934 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:50.967899    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"]
Nov 02 03:37:50.968398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:50.968086    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb" podUID="32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" containerName="maas-api" containerID="cri-o://29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946" gracePeriod=30
Nov 02 03:37:51.110045 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.110014    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:51.240685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.240659    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:37:51.291105 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.291077    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls\") pod \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") "
Nov 02 03:37:51.291255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.291126    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-openshift-service-ca\") pod \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") "
Nov 02 03:37:51.291255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.291158    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-metrics-certs\") pod \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") "
Nov 02 03:37:51.291255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.291233    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-kube-api-access-bjfpd\" (UniqueName: \"kubernetes.io/projected/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-kube-api-access-bjfpd\") pod \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\" (UID: \"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a\") "
Nov 02 03:37:51.291533 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.291502    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-openshift-service-ca" pod "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" (UID: "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:37:51.291836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.291815    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:51.293618 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.293594    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-metrics-certs" pod "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" (UID: "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:37:51.293618 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.293605    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-kube-api-access-bjfpd" pod "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" (UID: "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a"). InnerVolumeSpecName "kube-api-access-bjfpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:37:51.293799 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.293696    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls" pod "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" (UID: "32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:37:51.393120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.393087    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:51.393120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.393118    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:51.393321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.393144    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjfpd\" (UniqueName: \"kubernetes.io/projected/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a-kube-api-access-bjfpd\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:51.473350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.473275    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"]
Nov 02 03:37:51.473514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.473472    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v" podUID="12ff28d1-a14f-4641-a567-a4aa90b65af5" containerName="payload-processing" containerID="cri-o://3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d" gracePeriod=30
Nov 02 03:37:51.736735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.736713    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:37:51.797766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.797738    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12ff28d1-a14f-4641-a567-a4aa90b65af5-kube-api-access-4ftf6\" (UniqueName: \"kubernetes.io/projected/12ff28d1-a14f-4641-a567-a4aa90b65af5-kube-api-access-4ftf6\") pod \"12ff28d1-a14f-4641-a567-a4aa90b65af5\" (UID: \"12ff28d1-a14f-4641-a567-a4aa90b65af5\") "
Nov 02 03:37:51.797931 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.797890    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12ff28d1-a14f-4641-a567-a4aa90b65af5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ff28d1-a14f-4641-a567-a4aa90b65af5-plugins-config-volume\") pod \"12ff28d1-a14f-4641-a567-a4aa90b65af5\" (UID: \"12ff28d1-a14f-4641-a567-a4aa90b65af5\") "
Nov 02 03:37:51.798226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.798206    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12ff28d1-a14f-4641-a567-a4aa90b65af5-plugins-config-volume" pod "12ff28d1-a14f-4641-a567-a4aa90b65af5" (UID: "12ff28d1-a14f-4641-a567-a4aa90b65af5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:37:51.798323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.798311    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ff28d1-a14f-4641-a567-a4aa90b65af5-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:51.800048 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.800030    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ff28d1-a14f-4641-a567-a4aa90b65af5-kube-api-access-4ftf6" pod "12ff28d1-a14f-4641-a567-a4aa90b65af5" (UID: "12ff28d1-a14f-4641-a567-a4aa90b65af5"). InnerVolumeSpecName "kube-api-access-4ftf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:37:51.899613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:51.899583    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ftf6\" (UniqueName: \"kubernetes.io/projected/12ff28d1-a14f-4641-a567-a4aa90b65af5-kube-api-access-4ftf6\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:37:52.071497 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.071463    4407 generic.go:356] "Generic (PLEG): container finished" podID="32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" containerID="29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946" exitCode=0
Nov 02 03:37:52.071998 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.071512    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"
Nov 02 03:37:52.071998 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.071549    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb" event={"ID":"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a","Type":"ContainerDied","Data":"29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946"}
Nov 02 03:37:52.071998 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.071601    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb" event={"ID":"32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a","Type":"ContainerDied","Data":"c038384789f1e11e0a4c335e0e980c9e39866e791adae927a048348234ca4313"}
Nov 02 03:37:52.071998 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.071650    4407 scope.go:122] "RemoveContainer" containerID="29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946"
Nov 02 03:37:52.072983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.072964    4407 generic.go:356] "Generic (PLEG): container finished" podID="12ff28d1-a14f-4641-a567-a4aa90b65af5" containerID="3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d" exitCode=0
Nov 02 03:37:52.073047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.072994    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v" event={"ID":"12ff28d1-a14f-4641-a567-a4aa90b65af5","Type":"ContainerDied","Data":"3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d"}
Nov 02 03:37:52.073047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.073013    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"
Nov 02 03:37:52.073047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.073025    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v" event={"ID":"12ff28d1-a14f-4641-a567-a4aa90b65af5","Type":"ContainerDied","Data":"48034d149646a59b4341e2f08a58f32ac2a070df4f6e3de648d476786f164fcd"}
Nov 02 03:37:52.085687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.085657    4407 scope.go:122] "RemoveContainer" containerID="29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946"
Nov 02 03:37:52.086082 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:52.086055    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946\": container with ID starting with 29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946 not found: ID does not exist" containerID="29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946"
Nov 02 03:37:52.086189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.086089    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946"} err="failed to get container status \"29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946\": rpc error: code = NotFound desc = could not find container \"29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946\": container with ID starting with 29aedcc91f356ff2b705d42450446483c20e800cc3aeffb10624d56be1b6e946 not found: ID does not exist"
Nov 02 03:37:52.086189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.086107    4407 scope.go:122] "RemoveContainer" containerID="3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d"
Nov 02 03:37:52.094925 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.094905    4407 scope.go:122] "RemoveContainer" containerID="3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d"
Nov 02 03:37:52.095191 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:37:52.095174    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d\": container with ID starting with 3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d not found: ID does not exist" containerID="3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d"
Nov 02 03:37:52.095242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.095198    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d"} err="failed to get container status \"3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d\": rpc error: code = NotFound desc = could not find container \"3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d\": container with ID starting with 3c624d79bf4d67224ba6961ca4d32b6b6ef3f7b17f239fb775306d79f4835a1d not found: ID does not exist"
Nov 02 03:37:52.241159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.241065    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"]
Nov 02 03:37:52.244369 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.244344    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-018375-999b4f876-v4d8v"]
Nov 02 03:37:52.351597 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.351561    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12ff28d1-a14f-4641-a567-a4aa90b65af5" path="/var/lib/kubelet/pods/12ff28d1-a14f-4641-a567-a4aa90b65af5/volumes"
Nov 02 03:37:52.353915 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.353891    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"]
Nov 02 03:37:52.357902 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.357876    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-018375-579b795d4b-pw5fb"]
Nov 02 03:37:52.779048 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:52.779016    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:53.062284 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:53.062198    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:53.225336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:53.225303    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:53.720498 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:53.720459    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:53.880720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:53.880690    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:54.352267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:54.352230    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" path="/var/lib/kubelet/pods/32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a/volumes"
Nov 02 03:37:54.919818 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:54.919783    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:55.385052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385018    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"]
Nov 02 03:37:55.385510 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385488    4407 cpu_manager.go:414] "containerMap: removing container" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy"
Nov 02 03:37:55.385561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385510    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy"
Nov 02 03:37:55.385561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385529    4407 cpu_manager.go:414] "containerMap: removing container" podUID="aa21aeaa-e46e-4090-b212-1d0bc2f608db" containerName="maas-api"
Nov 02 03:37:55.385561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385537    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa21aeaa-e46e-4090-b212-1d0bc2f608db" containerName="maas-api"
Nov 02 03:37:55.385561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385547    4407 cpu_manager.go:414] "containerMap: removing container" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy"
Nov 02 03:37:55.385561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385555    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy"
Nov 02 03:37:55.385561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385565    4407 cpu_manager.go:414] "containerMap: removing container" podUID="924714d8-000e-4584-b58e-d9631ac77a20" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385569    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="924714d8-000e-4584-b58e-d9631ac77a20" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385577    4407 cpu_manager.go:414] "containerMap: removing container" podUID="083082f5-1ba8-46ea-a8e7-e0d588a66a42" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385581    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="083082f5-1ba8-46ea-a8e7-e0d588a66a42" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385589    4407 cpu_manager.go:414] "containerMap: removing container" podUID="12ff28d1-a14f-4641-a567-a4aa90b65af5" containerName="payload-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385595    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12ff28d1-a14f-4641-a567-a4aa90b65af5" containerName="payload-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385610    4407 cpu_manager.go:414] "containerMap: removing container" podUID="d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d" containerName="payload-pre-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385615    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d" containerName="payload-pre-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385657    4407 cpu_manager.go:414] "containerMap: removing container" podUID="13721185-dc40-47f5-99ac-e0d3be42cf49" containerName="payload-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385662    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13721185-dc40-47f5-99ac-e0d3be42cf49" containerName="payload-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385670    4407 cpu_manager.go:414] "containerMap: removing container" podUID="32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385675    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385687    4407 cpu_manager.go:414] "containerMap: removing container" podUID="ecf9b24a-128d-4247-92c0-e8b70113e800" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385692    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ecf9b24a-128d-4247-92c0-e8b70113e800" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385791    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="924714d8-000e-4584-b58e-d9631ac77a20" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385801    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="083082f5-1ba8-46ea-a8e7-e0d588a66a42" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385807    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="12ff28d1-a14f-4641-a567-a4aa90b65af5" containerName="payload-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385817    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9cac2e3-1ff2-4c5e-8b24-4537afb8dd4d" containerName="payload-pre-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385824    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="13721185-dc40-47f5-99ac-e0d3be42cf49" containerName="payload-processing"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385830    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="32bb14ba-5bb5-4651-ad64-4c9d93bb4b1a" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385836    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="ecf9b24a-128d-4247-92c0-e8b70113e800" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385847    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385856    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa21aeaa-e46e-4090-b212-1d0bc2f608db" containerName="maas-api"
Nov 02 03:37:55.385906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.385862    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="06c937a7-e688-49e8-ab6e-6726e3afa7a9" containerName="istio-proxy"
Nov 02 03:37:55.390520 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.390502    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.483720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.483676    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-3d563c-openshift-default-dockercfg-dnvqw\""
Nov 02 03:37:55.486504 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.486441    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021d20bc-9467-46c6-8705-7b336047a7fa-istio-podinfo\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.486504 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.486493    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-envoy\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.486804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.486572    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-credential-socket\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.486804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.486657    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-istio-token\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.486804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.486735    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcw9b\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-kube-api-access-gcw9b\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.486974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.486870    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021d20bc-9467-46c6-8705-7b336047a7fa-istiod-ca-cert\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.486974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.486939    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-certs\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.487075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.486972    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-socket\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.487075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.487012    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-data\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.497806 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.497767    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:55.539505 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.539469    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"]
Nov 02 03:37:55.588345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588309    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-credential-socket\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588395    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-istio-token\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588428    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcw9b\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-kube-api-access-gcw9b\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588568    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021d20bc-9467-46c6-8705-7b336047a7fa-istiod-ca-cert\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588683    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-certs\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588720    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-socket\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588767    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-data\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588820 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588796    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-credential-socket\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588874    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021d20bc-9467-46c6-8705-7b336047a7fa-istio-podinfo\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.588979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.588913    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-envoy\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.589081 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.589016    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-certs\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.589131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.589096    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-socket\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.589131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.589122    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-data\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.589324 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.589300    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021d20bc-9467-46c6-8705-7b336047a7fa-istiod-ca-cert\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.591388 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.591366    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-envoy\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.591596 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.591576    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021d20bc-9467-46c6-8705-7b336047a7fa-istio-podinfo\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.646539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.646476    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-istio-token\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.646713 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.646594    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcw9b\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-kube-api-access-gcw9b\") pod \"e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") " pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.703807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.703774    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:55.907057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:55.906975    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:56.140929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:56.140904    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"]
Nov 02 03:37:56.141450 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:37:56.141426    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021d20bc_9467_46c6_8705_7b336047a7fa.slice/crio-43936d8e062c698b473e63cb88341b36c70ff2a38b54c0f7053c8924251eb123 WatchSource:0}: Error finding container 43936d8e062c698b473e63cb88341b36c70ff2a38b54c0f7053c8924251eb123: Status 404 returned error can't find the container with id 43936d8e062c698b473e63cb88341b36c70ff2a38b54c0f7053c8924251eb123
Nov 02 03:37:56.143859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:56.143827    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:37:56.143944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:56.143891    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:37:56.143944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:56.143918    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:37:57.104040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:57.104006    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" event={"ID":"021d20bc-9467-46c6-8705-7b336047a7fa","Type":"ContainerStarted","Data":"5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791"}
Nov 02 03:37:57.104443 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:57.104060    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" event={"ID":"021d20bc-9467-46c6-8705-7b336047a7fa","Type":"ContainerStarted","Data":"43936d8e062c698b473e63cb88341b36c70ff2a38b54c0f7053c8924251eb123"}
Nov 02 03:37:57.223950 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:57.223900    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" podStartSLOduration=2.223885655 podStartE2EDuration="2.223885655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:37:57.22131653 +0000 UTC m=+3021.016665094" watchObservedRunningTime="2026-11-02 03:37:57.223885655 +0000 UTC m=+3021.019234221"
Nov 02 03:37:57.704193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:57.704158    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:57.709775 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:57.709749    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:58.109513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:58.109469    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:58.110438 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:58.110415    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:37:58.387113 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:58.387031    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:59.583788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:59.583753    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:59.753712 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:59.753683    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:37:59.923334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:37:59.923248    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:00.086606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:00.086572    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:00.250576 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:00.250551    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:01.308325 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:01.308292    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:02.235084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.235048    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"]
Nov 02 03:38:02.239200 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.239179    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272048 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272005    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbbxb\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-kube-api-access-fbbxb\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272071    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-data\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272134    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/049b989c-9dbc-4cc0-8177-a3d83778296f-istiod-ca-cert\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272164    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-envoy\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272255 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272187    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-podinfo\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272320    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-socket\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272373    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-certs\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272458    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-credential-socket\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.272671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.272496    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-token\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.302568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.302541    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-2912e6-openshift-default-dockercfg-77gp6\""
Nov 02 03:38:02.359689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.359653    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"]
Nov 02 03:38:02.373453 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373415    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-token\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.373644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373487    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbbxb\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-kube-api-access-fbbxb\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.373644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373527    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-data\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.373644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373549    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/049b989c-9dbc-4cc0-8177-a3d83778296f-istiod-ca-cert\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.373644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373566    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-envoy\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.373644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373588    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-podinfo\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.373960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373663    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-socket\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.373960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373716    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-certs\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.373960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.373783    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-credential-socket\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.374121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.374006    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-data\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.374412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.374360    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-socket\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.374594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.374569    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-credential-socket\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.374791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.374767    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-certs\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.374917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.374893    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/049b989c-9dbc-4cc0-8177-a3d83778296f-istiod-ca-cert\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.376206 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.376183    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-envoy\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.376567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.376544    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-podinfo\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.423924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.423890    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:02.434472 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.434444    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-token\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.436393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.436360    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbbxb\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-kube-api-access-fbbxb\") pod \"e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") " pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.553666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.553615    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:02.590341 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:02.590301    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:03.002975 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.002945    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"]
Nov 02 03:38:03.004040 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:38:03.004011    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049b989c_9dbc_4cc0_8177_a3d83778296f.slice/crio-7966085c5c82f0f775048b5bf3a8345f43b48b0b7bc08dcbdc59780cbb1d0ee3 WatchSource:0}: Error finding container 7966085c5c82f0f775048b5bf3a8345f43b48b0b7bc08dcbdc59780cbb1d0ee3: Status 404 returned error can't find the container with id 7966085c5c82f0f775048b5bf3a8345f43b48b0b7bc08dcbdc59780cbb1d0ee3
Nov 02 03:38:03.006595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.006561    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:38:03.006685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.006651    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:38:03.006726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.006685    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:38:03.139243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.139205    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" event={"ID":"049b989c-9dbc-4cc0-8177-a3d83778296f","Type":"ContainerStarted","Data":"cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12"}
Nov 02 03:38:03.139444 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.139269    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" event={"ID":"049b989c-9dbc-4cc0-8177-a3d83778296f","Type":"ContainerStarted","Data":"7966085c5c82f0f775048b5bf3a8345f43b48b0b7bc08dcbdc59780cbb1d0ee3"}
Nov 02 03:38:03.264708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.264589    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" podStartSLOduration=1.264575792 podStartE2EDuration="1.264575792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:38:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:38:03.262738692 +0000 UTC m=+3027.058087258" watchObservedRunningTime="2026-11-02 03:38:03.264575792 +0000 UTC m=+3027.059924357"
Nov 02 03:38:03.281116 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.281092    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:03.445858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.445817    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:03.559687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.554510    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:03.731655 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:03.731589    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:04.560154 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:04.560126    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:05.150751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:05.150720    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:05.151694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:05.151673    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:38:05.192009 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:38:05.191972    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:38:06.513732 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:06.513699    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:06.677768 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:06.677741    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:06.847043 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:06.846966    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:07.146498 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:07.146399    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:08.272825 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.272789    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:08.732098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.732019    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"]
Nov 02 03:38:08.736296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.736274    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.815956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.815927    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-cb56e9-openshift-default-dockercfg-vt2lm\""
Nov 02 03:38:08.850955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.850891    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:08.851136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.850973    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-socket\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.851136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.851095    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr6h8\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-kube-api-access-tr6h8\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.851136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.851132    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c58f9d-2706-4af2-8007-252410af7891-istiod-ca-cert\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.851136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.851173    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c58f9d-2706-4af2-8007-252410af7891-istio-podinfo\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.851363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.851217    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-data\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.851363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.851282    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-credential-socket\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.851363 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.851360    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-envoy\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.851475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.851400    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-certs\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.851475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.851430    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-istio-token\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.869459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.869422    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"]
Nov 02 03:38:08.952376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.952337    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr6h8\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-kube-api-access-tr6h8\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.952376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.952375    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c58f9d-2706-4af2-8007-252410af7891-istiod-ca-cert\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.952613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.952401    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c58f9d-2706-4af2-8007-252410af7891-istio-podinfo\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.952613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.952536    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-data\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.952741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.952680    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-credential-socket\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.952810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.952790    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-envoy\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.952990 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.952967    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-data\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.953099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.953016    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-certs\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.953434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.953115    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-istio-token\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.953434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.953025    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-credential-socket\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.953434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.953166    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-socket\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.953434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.953179    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c58f9d-2706-4af2-8007-252410af7891-istiod-ca-cert\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.953434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.953375    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-certs\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.953756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.953594    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-socket\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.955590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.955569    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-envoy\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:08.955832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:08.955816    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c58f9d-2706-4af2-8007-252410af7891-istio-podinfo\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:09.010772 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.010739    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-istio-token\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:09.010968 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.010789    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr6h8\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-kube-api-access-tr6h8\") pod \"e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") " pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:09.049890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.049861    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:09.089346 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.089320    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:09.245157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.245125    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:09.476968 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.476938    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"]
Nov 02 03:38:09.477783 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:38:09.477758    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c58f9d_2706_4af2_8007_252410af7891.slice/crio-d6b394ff5a941b234d2650c7fa16ca89d037e3604c6c8f0516bc8fb34a398797 WatchSource:0}: Error finding container d6b394ff5a941b234d2650c7fa16ca89d037e3604c6c8f0516bc8fb34a398797: Status 404 returned error can't find the container with id d6b394ff5a941b234d2650c7fa16ca89d037e3604c6c8f0516bc8fb34a398797
Nov 02 03:38:09.480361 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.480325    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:38:09.480460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.480404    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:38:09.480460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:09.480448    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:38:10.183343 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:10.183310    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" event={"ID":"25c58f9d-2706-4af2-8007-252410af7891","Type":"ContainerStarted","Data":"90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80"}
Nov 02 03:38:10.183526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:10.183359    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" event={"ID":"25c58f9d-2706-4af2-8007-252410af7891","Type":"ContainerStarted","Data":"d6b394ff5a941b234d2650c7fa16ca89d037e3604c6c8f0516bc8fb34a398797"}
Nov 02 03:38:10.302211 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:10.302161    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podStartSLOduration=2.302146955 podStartE2EDuration="2.302146955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:38:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:38:10.300364693 +0000 UTC m=+3034.095713257" watchObservedRunningTime="2026-11-02 03:38:10.302146955 +0000 UTC m=+3034.097495520"
Nov 02 03:38:10.395971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:10.395936    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:10.581733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:10.581704    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:11.050719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:11.050690    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:11.056450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:11.056427    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:11.189077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:11.189043    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:11.190090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:11.190072    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:38:13.168839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:13.168808    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:13.345071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:13.345039    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:13.504413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:13.504381    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:14.366710 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:14.366676    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:14.684392 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:14.684304    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:17.384174 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:17.384143    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:17.546736 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:17.546704    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:19.501720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:19.501478    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:24.309982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:24.309947    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:24.504199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:24.504161    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:27.314908 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:27.314877    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:28.437065 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:28.437030    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:29.014980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:29.014942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:29.307071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:29.306964    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:29.477332 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:29.477304    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:32.767608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:32.767576    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:32.945199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:32.945167    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:33.112955 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:33.112877    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:34.346819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:34.346778    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:35.189276 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:38:35.189243    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:38:35.228721 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:35.228690    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:38:36.239567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.239535    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:36.412840 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.412813    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:36.627914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.627874    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"]
Nov 02 03:38:36.633267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.633244    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:36.688249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.688225    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-1b1008\""
Nov 02 03:38:36.739328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.739293    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-1b1008-dockercfg-grtbl\""
Nov 02 03:38:36.743012 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.742977    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"]
Nov 02 03:38:36.743280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.743217    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f182996f-880d-4999-a72d-9f72183578e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc\" (UID: \"f182996f-880d-4999-a72d-9f72183578e5\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:36.743488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.743453    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwz6f\" (UniqueName: \"kubernetes.io/projected/f182996f-880d-4999-a72d-9f72183578e5-kube-api-access-mwz6f\") pod \"payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc\" (UID: \"f182996f-880d-4999-a72d-9f72183578e5\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:36.795006 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.794963    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"]
Nov 02 03:38:36.802332 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.802303    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:36.844495 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.844466    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwz6f\" (UniqueName: \"kubernetes.io/projected/f182996f-880d-4999-a72d-9f72183578e5-kube-api-access-mwz6f\") pod \"payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc\" (UID: \"f182996f-880d-4999-a72d-9f72183578e5\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:36.844726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.844703    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f182996f-880d-4999-a72d-9f72183578e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc\" (UID: \"f182996f-880d-4999-a72d-9f72183578e5\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:36.845356 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.845337    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f182996f-880d-4999-a72d-9f72183578e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc\" (UID: \"f182996f-880d-4999-a72d-9f72183578e5\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:36.854302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.854265    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"]
Nov 02 03:38:36.901715 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.901649    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwz6f\" (UniqueName: \"kubernetes.io/projected/f182996f-880d-4999-a72d-9f72183578e5-kube-api-access-mwz6f\") pod \"payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc\" (UID: \"f182996f-880d-4999-a72d-9f72183578e5\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:36.946283 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.946251    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qnsl\" (UniqueName: \"kubernetes.io/projected/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-kube-api-access-4qnsl\") pod \"payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb\" (UID: \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:36.946442 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.946326    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb\" (UID: \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:36.947024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:36.947005    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:37.048129 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:37.048097    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qnsl\" (UniqueName: \"kubernetes.io/projected/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-kube-api-access-4qnsl\") pod \"payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb\" (UID: \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:37.048329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:37.048182    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb\" (UID: \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:37.048942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:37.048920    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb\" (UID: \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:37.105819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:37.105788    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qnsl\" (UniqueName: \"kubernetes.io/projected/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-kube-api-access-4qnsl\") pod \"payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb\" (UID: \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:37.119697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:37.119667    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:37.327453 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:37.327428    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"]
Nov 02 03:38:37.328787 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:38:37.328758    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf182996f_880d_4999_a72d_9f72183578e5.slice/crio-fd75e8b1b7cd0686472eb8f8848b94307e128c91fcff89b347ebe99a30fee6df WatchSource:0}: Error finding container fd75e8b1b7cd0686472eb8f8848b94307e128c91fcff89b347ebe99a30fee6df: Status 404 returned error can't find the container with id fd75e8b1b7cd0686472eb8f8848b94307e128c91fcff89b347ebe99a30fee6df
Nov 02 03:38:37.356104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:37.356076    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc" event={"ID":"f182996f-880d-4999-a72d-9f72183578e5","Type":"ContainerStarted","Data":"fd75e8b1b7cd0686472eb8f8848b94307e128c91fcff89b347ebe99a30fee6df"}
Nov 02 03:38:37.495672 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:37.495649    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"]
Nov 02 03:38:37.496522 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:38:37.496501    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb99f1f_a3fd_49d1_9e5b_468d73aac84a.slice/crio-021f5a5792f38a3957d230d45e1d57d06c9066b173e269c2aba3ecd50f1d482b WatchSource:0}: Error finding container 021f5a5792f38a3957d230d45e1d57d06c9066b173e269c2aba3ecd50f1d482b: Status 404 returned error can't find the container with id 021f5a5792f38a3957d230d45e1d57d06c9066b173e269c2aba3ecd50f1d482b
Nov 02 03:38:38.363865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:38.363824    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc" event={"ID":"f182996f-880d-4999-a72d-9f72183578e5","Type":"ContainerStarted","Data":"dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc"}
Nov 02 03:38:38.364345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:38.363989    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:38.365470 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:38.365444    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb" event={"ID":"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a","Type":"ContainerStarted","Data":"59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041"}
Nov 02 03:38:38.365614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:38.365486    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb" event={"ID":"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a","Type":"ContainerStarted","Data":"021f5a5792f38a3957d230d45e1d57d06c9066b173e269c2aba3ecd50f1d482b"}
Nov 02 03:38:38.365614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:38.365524    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:38.479682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:38.479610    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc" podStartSLOduration=2.479596315 podStartE2EDuration="2.479596315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:38:38.47783523 +0000 UTC m=+3062.273183796" watchObservedRunningTime="2026-11-02 03:38:38.479596315 +0000 UTC m=+3062.274944879"
Nov 02 03:38:38.595068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:38.594552    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb" podStartSLOduration=2.594534231 podStartE2EDuration="2.594534231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:38:38.592200273 +0000 UTC m=+3062.387548839" watchObservedRunningTime="2026-11-02 03:38:38.594534231 +0000 UTC m=+3062.389882798"
Nov 02 03:38:39.392344 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:39.392302    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:44.505956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:44.505920    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:45.523567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:45.523520    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:47.832246 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:47.832200    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:47.982329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:47.982300    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:49.374205 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:49.374178    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:38:49.374671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:49.374278    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:38:49.432112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:49.432083    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:49.591794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:49.591759    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:54.308483 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:54.308448    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:59.122875 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:59.122836    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:59.284427 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:59.284390    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:38:59.461006 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:38:59.460920    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:02.761874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:02.761840    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:39:04.051874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:04.051827    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:05.188715 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:39:05.188680    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:39:08.923478 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:08.923447    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:09.080819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:09.080790    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:09.236736 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:09.236656    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:14.040853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:14.040814    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:18.761681 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:18.761560    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:18.930296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:18.930263    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:19.082436 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:19.082327    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:19.581333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:19.581292    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:19.769345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:19.769311    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:20.836355 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:20.836316    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:21.025460 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:21.025421    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:23.796846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:23.796813    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:28.553800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:28.553710    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:32.718267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:32.718237    4407 generic.go:356] "Generic (PLEG): container finished" podID="021d20bc-9467-46c6-8705-7b336047a7fa" containerID="5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791" exitCode=0
Nov 02 03:39:32.718789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:32.718305    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" event={"ID":"021d20bc-9467-46c6-8705-7b336047a7fa","Type":"ContainerDied","Data":"5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791"}
Nov 02 03:39:32.718789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:32.718758    4407 scope.go:122] "RemoveContainer" containerID="5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791"
Nov 02 03:39:32.720425 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:32.720397    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:32.720498 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:32.720469    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:32.720538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:32.720509    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:33.294512 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:33.294473    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:33.729454 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:33.729356    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" event={"ID":"021d20bc-9467-46c6-8705-7b336047a7fa","Type":"ContainerStarted","Data":"1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568"}
Nov 02 03:39:34.705836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:34.705800    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:39:34.706068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:34.706042    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 03:39:34.706128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:34.706102    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 03:39:35.210869 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:39:35.210782    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:39:35.706045 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:35.706004    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 03:39:35.706260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:35.706072    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 03:39:36.706046 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:36.706011    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 03:39:36.706422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:36.706066    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 03:39:37.706086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:37.706048    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 03:39:37.706736 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:37.706700    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 03:39:38.109647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:38.109449    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:38.704500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:38.704460    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 03:39:38.704705 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:38.704528    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 03:39:39.708966 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:39.708936    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:39:39.709391 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:39.709202    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:39:39.709928 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:39.709910    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:39:41.010391 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:41.010360    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:42.030914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.030873    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:42.503783 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.503742    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"]
Nov 02 03:39:42.506511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.506494    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.625777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.625745    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"]
Nov 02 03:39:42.626610 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.626577    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:42.627406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627378    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-9b06f8-openshift-default-dockercfg-5z2m7\""
Nov 02 03:39:42.627466 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627387    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1946862-d724-4984-a6a7-2a44608f1658-istio-podinfo\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.627511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627467    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-credential-socket\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.627556 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627528    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-certs\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.627556 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627548    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-socket\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.627657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627590    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-data\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.627708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627674    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s5j9\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-kube-api-access-9s5j9\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.627767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627714    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-istio-token\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.627767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627736    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-envoy\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.627767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.627754    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1946862-d724-4984-a6a7-2a44608f1658-istiod-ca-cert\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.728917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.728884    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-credential-socket\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.728960    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-certs\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.728980    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-socket\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729020    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-data\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729072    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s5j9\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-kube-api-access-9s5j9\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729103    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-istio-token\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729120    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-envoy\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729142    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1946862-d724-4984-a6a7-2a44608f1658-istiod-ca-cert\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729204    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1946862-d724-4984-a6a7-2a44608f1658-istio-podinfo\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729353    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-credential-socket\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729402    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-certs\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729525    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-data\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729714    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-socket\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.729900 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.729880    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1946862-d724-4984-a6a7-2a44608f1658-istiod-ca-cert\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.731562 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.731540    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-envoy\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.731847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.731831    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1946862-d724-4984-a6a7-2a44608f1658-istio-podinfo\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.787153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.787079    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-istio-token\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.787274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.787193    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s5j9\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-kube-api-access-9s5j9\") pod \"e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") " pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.818838 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.818810    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:42.972345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:42.972309    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:43.130003 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.129912    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:43.251153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.251126    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"]
Nov 02 03:39:43.252398 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:39:43.252377    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1946862_d724_4984_a6a7_2a44608f1658.slice/crio-e57dce7d2ff02d908c0dd87fd87bf4567101e4bcc95d61f9c9b9d686b51c586c WatchSource:0}: Error finding container e57dce7d2ff02d908c0dd87fd87bf4567101e4bcc95d61f9c9b9d686b51c586c: Status 404 returned error can't find the container with id e57dce7d2ff02d908c0dd87fd87bf4567101e4bcc95d61f9c9b9d686b51c586c
Nov 02 03:39:43.254916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.254882    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:43.254993 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.254950    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:43.254993 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.254981    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:43.282014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.281988    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:43.434562 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.434476    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:43.787309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.787274    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" event={"ID":"d1946862-d724-4984-a6a7-2a44608f1658","Type":"ContainerStarted","Data":"2a6b8d39282b751712dd4b7e1cef33084871fc4b5e2b8420822ec08b71cfdf63"}
Nov 02 03:39:43.787493 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.787327    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" event={"ID":"d1946862-d724-4984-a6a7-2a44608f1658","Type":"ContainerStarted","Data":"e57dce7d2ff02d908c0dd87fd87bf4567101e4bcc95d61f9c9b9d686b51c586c"}
Nov 02 03:39:43.819944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.819912    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:43.906830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:43.906749    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" podStartSLOduration=1.9067334040000001 podStartE2EDuration="1.906733404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:39:43.905786747 +0000 UTC m=+3127.701135321" watchObservedRunningTime="2026-11-02 03:39:43.906733404 +0000 UTC m=+3127.702081968"
Nov 02 03:39:44.825427 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:44.825398    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:45.798754 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:45.798718    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:45.799805 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:45.799784    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:39:45.864672 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:45.864643    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:46.667762 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:46.667728    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:46.849811 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:46.849778    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:47.003590 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:47.003564    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:48.495939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:48.495901    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:39:48.523461 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:48.523430    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:51.521650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:51.518218    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:51.667956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:51.667849    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:53.730603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:53.730553    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:57.231924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:57.231882    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:39:57.232474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:57.231964    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:39:58.560140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:58.560106    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:39:59.896660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.896592    4407 generic.go:356] "Generic (PLEG): container finished" podID="25c58f9d-2706-4af2-8007-252410af7891" containerID="90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80" exitCode=0
Nov 02 03:39:59.897201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.896662    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" event={"ID":"25c58f9d-2706-4af2-8007-252410af7891","Type":"ContainerDied","Data":"90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80"}
Nov 02 03:39:59.897201 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.897105    4407 scope.go:122] "RemoveContainer" containerID="90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80"
Nov 02 03:39:59.898544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.898518    4407 generic.go:356] "Generic (PLEG): container finished" podID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerID="cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12" exitCode=0
Nov 02 03:39:59.898669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.898552    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" event={"ID":"049b989c-9dbc-4cc0-8177-a3d83778296f","Type":"ContainerDied","Data":"cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12"}
Nov 02 03:39:59.898773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.898745    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:59.898827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.898814    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:59.898862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.898851    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:59.898903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.898891    4407 scope.go:122] "RemoveContainer" containerID="cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12"
Nov 02 03:39:59.900159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.900132    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:59.900226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.900195    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:39:59.900268 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:39:59.900226    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:40:00.906236 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:00.906197    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" event={"ID":"049b989c-9dbc-4cc0-8177-a3d83778296f","Type":"ContainerStarted","Data":"b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245"}
Nov 02 03:40:00.908200 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:00.908172    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" event={"ID":"25c58f9d-2706-4af2-8007-252410af7891","Type":"ContainerStarted","Data":"998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542"}
Nov 02 03:40:01.050466 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:01.050434    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:40:01.050698 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:01.050678    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 02 03:40:01.050753 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:01.050725    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 02 03:40:01.554812 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:01.554776    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:40:01.556225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:01.556198    4407 patch_prober.go:29] interesting pod/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 03:40:01.556350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:01.556247    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 03:40:02.050350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:02.050310    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 02 03:40:02.050808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:02.050369    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 02 03:40:02.554250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:02.554221    4407 patch_prober.go:29] interesting pod/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 03:40:02.554513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:02.554491    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 03:40:03.050311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:03.050279    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 02 03:40:03.050490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:03.050336    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 02 03:40:03.323708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:03.322989    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:03.554853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:03.554821    4407 patch_prober.go:29] interesting pod/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 03:40:03.555038 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:03.554886    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 03:40:04.051039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:04.051009    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 02 03:40:04.051451 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:04.051061    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 02 03:40:04.554099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:04.554068    4407 patch_prober.go:29] interesting pod/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 03:40:04.554286 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:04.554120    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 03:40:05.050966 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:05.050932    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 02 03:40:05.051212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:05.051013    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 02 03:40:05.151465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:05.151425    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:40:05.277686 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:40:05.277580    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:40:05.554910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:05.554875    4407 patch_prober.go:29] interesting pod/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 03:40:05.555089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:05.554929    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 03:40:06.056068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:06.056028    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 02 03:40:06.056722 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:06.056679    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 02 03:40:06.554609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:06.554576    4407 patch_prober.go:29] interesting pod/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 03:40:06.554828 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:06.554657    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 03:40:07.130291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:07.130259    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:40:07.131119 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:07.131089    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:40:07.131945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:07.131913    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:40:07.558909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:07.558879    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:40:07.560069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:07.560046    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:40:08.094967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:08.094933    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:12.882911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:12.882883    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:17.667372 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:17.667332    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:22.082170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:22.082133    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:40:24.268837 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:24.268798    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:24.426906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:24.426873    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:27.824218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:27.824186    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:28.474517 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:28.474487    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:29.040671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:29.040609    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:30.126013 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:30.125976    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:31.367999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:31.367960    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:31.548774 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:31.548742    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:32.889887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:32.889851    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:35.189424 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:40:35.189383    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:40:35.628079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:35.628044    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:35.886543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:35.886458    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:36.049278 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:36.049245    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:38.189790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:38.189752    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:41.164755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:41.164725    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:43.264818 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:43.264785    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:43.426112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:43.426077    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:46.212161 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:46.212126    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:46.676813 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:46.676784    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:46.853430 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:46.853400    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:47.071086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.071052    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"]
Nov 02 03:40:47.074853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.074832    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:47.130874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.130838    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"]
Nov 02 03:40:47.183407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.183370    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qnkn\" (UniqueName: \"kubernetes.io/projected/c2230c72-35df-4e94-8759-273f34800d71-kube-api-access-5qnkn\") pod \"payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch\" (UID: \"c2230c72-35df-4e94-8759-273f34800d71\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:47.185612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.185575    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2230c72-35df-4e94-8759-273f34800d71-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch\" (UID: \"c2230c72-35df-4e94-8759-273f34800d71\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:47.186079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.184464    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-ad96f4-dockercfg-hmbrj\""
Nov 02 03:40:47.186310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.184335    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-ad96f4\""
Nov 02 03:40:47.240196 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.240162    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"]
Nov 02 03:40:47.244985 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.244960    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:47.286419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.286390    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"]
Nov 02 03:40:47.288098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.288068    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2230c72-35df-4e94-8759-273f34800d71-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch\" (UID: \"c2230c72-35df-4e94-8759-273f34800d71\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:47.288431 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.288329    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qnkn\" (UniqueName: \"kubernetes.io/projected/c2230c72-35df-4e94-8759-273f34800d71-kube-api-access-5qnkn\") pod \"payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch\" (UID: \"c2230c72-35df-4e94-8759-273f34800d71\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:47.289544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.289521    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2230c72-35df-4e94-8759-273f34800d71-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch\" (UID: \"c2230c72-35df-4e94-8759-273f34800d71\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:47.346093 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.346014    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qnkn\" (UniqueName: \"kubernetes.io/projected/c2230c72-35df-4e94-8759-273f34800d71-kube-api-access-5qnkn\") pod \"payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch\" (UID: \"c2230c72-35df-4e94-8759-273f34800d71\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:47.388240 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.388206    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:47.389706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.389680    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwh9g\" (UniqueName: \"kubernetes.io/projected/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-kube-api-access-wwh9g\") pod \"payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq\" (UID: \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:47.389857 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.389835    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq\" (UID: \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:47.491011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.490965    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwh9g\" (UniqueName: \"kubernetes.io/projected/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-kube-api-access-wwh9g\") pod \"payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq\" (UID: \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:47.491187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.491120    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq\" (UID: \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:47.492161 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.492072    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq\" (UID: \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:47.548769 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.548741    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwh9g\" (UniqueName: \"kubernetes.io/projected/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-kube-api-access-wwh9g\") pod \"payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq\" (UID: \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:47.559772 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.559749    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:47.759540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.759510    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"]
Nov 02 03:40:47.760384 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:40:47.760361    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2230c72_35df_4e94_8759_273f34800d71.slice/crio-63db99b19b90d112c48a8c9008e4e84bab1db3d484d1b710f5ee02158b5bad52 WatchSource:0}: Error finding container 63db99b19b90d112c48a8c9008e4e84bab1db3d484d1b710f5ee02158b5bad52: Status 404 returned error can't find the container with id 63db99b19b90d112c48a8c9008e4e84bab1db3d484d1b710f5ee02158b5bad52
Nov 02 03:40:47.938811 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:47.938783    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"]
Nov 02 03:40:47.939785 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:40:47.939759    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6269d59_e3bc_4fe0_acc9_02bbaeefd570.slice/crio-eac548c011991f4e9f9debbf31b20f67dc0bd1c08dfe07e82647cccb5cd64471 WatchSource:0}: Error finding container eac548c011991f4e9f9debbf31b20f67dc0bd1c08dfe07e82647cccb5cd64471: Status 404 returned error can't find the container with id eac548c011991f4e9f9debbf31b20f67dc0bd1c08dfe07e82647cccb5cd64471
Nov 02 03:40:48.222819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.222718    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq" event={"ID":"f6269d59-e3bc-4fe0-acc9-02bbaeefd570","Type":"ContainerStarted","Data":"b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563"}
Nov 02 03:40:48.222819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.222779    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq" event={"ID":"f6269d59-e3bc-4fe0-acc9-02bbaeefd570","Type":"ContainerStarted","Data":"eac548c011991f4e9f9debbf31b20f67dc0bd1c08dfe07e82647cccb5cd64471"}
Nov 02 03:40:48.222819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.222807    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:40:48.224279 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.224256    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch" event={"ID":"c2230c72-35df-4e94-8759-273f34800d71","Type":"ContainerStarted","Data":"1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e"}
Nov 02 03:40:48.224400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.224297    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch" event={"ID":"c2230c72-35df-4e94-8759-273f34800d71","Type":"ContainerStarted","Data":"63db99b19b90d112c48a8c9008e4e84bab1db3d484d1b710f5ee02158b5bad52"}
Nov 02 03:40:48.224400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.224326    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:48.337752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.337693    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq" podStartSLOduration=1.337674245 podStartE2EDuration="1.337674245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:40:48.336091415 +0000 UTC m=+3192.131439980" watchObservedRunningTime="2026-11-02 03:40:48.337674245 +0000 UTC m=+3192.133022806"
Nov 02 03:40:48.369710 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.369685    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:48.469718 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:48.469657    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch" podStartSLOduration=1.469616781 podStartE2EDuration="1.469616781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:40:48.467022544 +0000 UTC m=+3192.262371110" watchObservedRunningTime="2026-11-02 03:40:48.469616781 +0000 UTC m=+3192.264965347"
Nov 02 03:40:53.140390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:53.140359    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:54.995063 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:54.995029    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:55.147805 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:55.147762    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:57.897318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:57.897287    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:40:59.232874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:59.232846    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:40:59.233260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:40:59.232921    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:41:02.600019 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:02.599966    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:02.758328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:02.758279    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:02.906689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:02.906604    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:05.188811 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:41:05.188774    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:41:05.728324 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:05.728269    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:06.818529 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:06.818495    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:07.448163 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:07.448114    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:07.624711 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:07.624673    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:07.885197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:07.885168    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:11.842022 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:11.841990    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:12.015581 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:12.015547    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:12.180724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:12.180618    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:13.059999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:13.059971    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:15.030340 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.030295    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:15.201568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.201536    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:15.428165 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.428082    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"]
Nov 02 03:41:15.432176 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.432158    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:15.480519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.480490    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"]
Nov 02 03:41:15.526560 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.526530    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-f68c7e\""
Nov 02 03:41:15.572650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.544542    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7abfea6e-c9e2-4c60-9793-be526dcedfd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf\" (UID: \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:15.573266 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.573232    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74zqd\" (UniqueName: \"kubernetes.io/projected/7abfea6e-c9e2-4c60-9793-be526dcedfd4-kube-api-access-74zqd\") pod \"payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf\" (UID: \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:15.674496 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.674462    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7abfea6e-c9e2-4c60-9793-be526dcedfd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf\" (UID: \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:15.674832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.674569    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74zqd\" (UniqueName: \"kubernetes.io/projected/7abfea6e-c9e2-4c60-9793-be526dcedfd4-kube-api-access-74zqd\") pod \"payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf\" (UID: \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:15.675602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.675579    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7abfea6e-c9e2-4c60-9793-be526dcedfd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf\" (UID: \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:15.732305 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.732234    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74zqd\" (UniqueName: \"kubernetes.io/projected/7abfea6e-c9e2-4c60-9793-be526dcedfd4-kube-api-access-74zqd\") pod \"payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf\" (UID: \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:15.746161 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:15.746138    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:16.117839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:16.117811    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"]
Nov 02 03:41:16.118769 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:41:16.118743    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7abfea6e_c9e2_4c60_9793_be526dcedfd4.slice/crio-e13848fafe2ebf917bc3922e5b31d146d85742481418e680853c22a481c0647e WatchSource:0}: Error finding container e13848fafe2ebf917bc3922e5b31d146d85742481418e680853c22a481c0647e: Status 404 returned error can't find the container with id e13848fafe2ebf917bc3922e5b31d146d85742481418e680853c22a481c0647e
Nov 02 03:41:16.411153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:16.411072    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf" event={"ID":"7abfea6e-c9e2-4c60-9793-be526dcedfd4","Type":"ContainerStarted","Data":"373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa"}
Nov 02 03:41:16.411153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:16.411123    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf" event={"ID":"7abfea6e-c9e2-4c60-9793-be526dcedfd4","Type":"ContainerStarted","Data":"e13848fafe2ebf917bc3922e5b31d146d85742481418e680853c22a481c0647e"}
Nov 02 03:41:16.411412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:16.411237    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:16.526865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:16.526805    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf" podStartSLOduration=1.5267883759999998 podStartE2EDuration="1.526788376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:41:16.524928768 +0000 UTC m=+3220.320277324" watchObservedRunningTime="2026-11-02 03:41:16.526788376 +0000 UTC m=+3220.322136947"
Nov 02 03:41:17.504558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:17.504523    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:19.290499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:19.290472    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:19.457139 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:19.457098    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:19.620187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:19.620084    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:20.165608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:20.165578    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:23.906995 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:23.906966    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:25.498319 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:25.498285    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:25.759189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:25.759119    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:25.922690 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:25.922654    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:26.113378 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:26.113290    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:26.382879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:26.382794    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:26.623373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:26.623324    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:27.420522 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:27.420485    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:41:28.888947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:28.888911    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:41:31.013292 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:31.013256    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:32.214036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:32.214006    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:32.379932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:32.379904    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:32.558960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:32.558928    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:33.241551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:33.241507    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"]
Nov 02 03:41:33.242026 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:33.241816    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy" containerID="cri-o://d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b" gracePeriod=30
Nov 02 03:41:35.215399 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:41:35.215365    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:41:36.210422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:36.210391    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:38.525746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.525679    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:41:38.558081 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.558052    4407 generic.go:356] "Generic (PLEG): container finished" podID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerID="d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b" exitCode=0
Nov 02 03:41:38.558276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.558110    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"
Nov 02 03:41:38.558276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.558148    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" event={"ID":"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a","Type":"ContainerDied","Data":"d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b"}
Nov 02 03:41:38.558276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.558225    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87" event={"ID":"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a","Type":"ContainerDied","Data":"a44234b62d0955e74415c854aa03b85f6ec6e395009a891121bd0ae26dd4019b"}
Nov 02 03:41:38.558276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.558258    4407 scope.go:122] "RemoveContainer" containerID="d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b"
Nov 02 03:41:38.570116 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.570097    4407 scope.go:122] "RemoveContainer" containerID="d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b"
Nov 02 03:41:38.570428 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:41:38.570403    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b\": container with ID starting with d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b not found: ID does not exist" containerID="d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b"
Nov 02 03:41:38.570488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.570438    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b"} err="failed to get container status \"d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b\": rpc error: code = NotFound desc = could not find container \"d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b\": container with ID starting with d13bed2ab9a2e84752aec651d9fc179e0a8961869d1a10c4cbf580aa629bc65b not found: ID does not exist"
Nov 02 03:41:38.690606 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690567    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-credential-socket\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.690829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690663    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-data\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.690829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690704    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-certs\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.690829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690755    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-token\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-token\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.690829 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690804    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-socket\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.691175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690837    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istiod-ca-cert\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.691175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690904    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-credential-socket" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:41:38.691175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690915    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-envoy\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.691175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.690957    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-certs" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:41:38.691175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.691035    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-data" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:41:38.691175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.691100    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-podinfo\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.691175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.691119    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-socket" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:41:38.691175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.691173    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-kube-api-access-8hpb5\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-kube-api-access-8hpb5\") pod \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\" (UID: \"7a453c7c-8a6a-4171-8d2a-08217c1d8f7a\") "
Nov 02 03:41:38.691561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.691295    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istiod-ca-cert" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:41:38.692791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.692460    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.692791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.692484    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.692791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.692500    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.692791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.692551    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.692791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.692570    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.694336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.694301    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-podinfo" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:41:38.694456 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.694350    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-token" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:41:38.694709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.694689    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-envoy" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:41:38.695353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.695335    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-kube-api-access-8hpb5" pod "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" (UID: "7a453c7c-8a6a-4171-8d2a-08217c1d8f7a"). InnerVolumeSpecName "kube-api-access-8hpb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:41:38.793967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.793897    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.793967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.793923    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.793967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.793933    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hpb5\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-kube-api-access-8hpb5\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.793967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.793944    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:41:38.983360 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.983322    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"]
Nov 02 03:41:38.985947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:38.985923    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5lq87"]
Nov 02 03:41:39.103797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:39.103694    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:39.258535 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:39.258500    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:39.566049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:39.566017    4407 generic.go:356] "Generic (PLEG): container finished" podID="d1946862-d724-4984-a6a7-2a44608f1658" containerID="2a6b8d39282b751712dd4b7e1cef33084871fc4b5e2b8420822ec08b71cfdf63" exitCode=0
Nov 02 03:41:39.566521 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:39.566089    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" event={"ID":"d1946862-d724-4984-a6a7-2a44608f1658","Type":"ContainerDied","Data":"2a6b8d39282b751712dd4b7e1cef33084871fc4b5e2b8420822ec08b71cfdf63"}
Nov 02 03:41:39.566521 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:39.566517    4407 scope.go:122] "RemoveContainer" containerID="2a6b8d39282b751712dd4b7e1cef33084871fc4b5e2b8420822ec08b71cfdf63"
Nov 02 03:41:39.568181 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:39.568149    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:41:39.568289 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:39.568236    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:41:39.568355 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:39.568286    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:41:40.353548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:40.353507    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" path="/var/lib/kubelet/pods/7a453c7c-8a6a-4171-8d2a-08217c1d8f7a/volumes"
Nov 02 03:41:40.575785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:40.575752    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" event={"ID":"d1946862-d724-4984-a6a7-2a44608f1658","Type":"ContainerStarted","Data":"cdebd7263ac6dcd02dcca661bf14c6e02b782909a7d0069385feae000b5d7449"}
Nov 02 03:41:40.819947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:40.819911    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:41:40.820153 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:40.820136    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 02 03:41:40.820210 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:40.820192    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 02 03:41:41.171824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:41.171741    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:41.819490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:41.819457    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 02 03:41:41.819889 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:41.819507    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 02 03:41:42.820070 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:42.820037    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 02 03:41:42.820472 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:42.820114    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 02 03:41:43.819312 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:43.819277    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 02 03:41:43.819505 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:43.819330    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 02 03:41:44.819198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:44.819162    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 02 03:41:44.819593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:44.819211    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 02 03:41:45.207708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:45.207158    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:45.462484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:45.462388    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:45.799348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:45.799318    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:41:45.823960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:45.823934    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:41:46.582607 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:46.582570    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:46.635377 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:46.635344    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:41:47.023755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:47.023720    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:47.218681 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:47.218610    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:47.392544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:47.392456    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:51.516488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:51.516454    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:51.799065 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:51.798990    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:51.958578 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:51.958544    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:52.397298 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:52.397269    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:56.973034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:56.972998    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:58.086158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:58.086125    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:58.243000 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:58.242971    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:41:58.385997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:58.385924    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"]
Nov 02 03:41:58.386140 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:58.386105    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy" containerID="cri-o://998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542" gracePeriod=30
Nov 02 03:41:58.417718 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:41:58.417690    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:00.939891 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:00.939852    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:42:01.720021 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:01.719986    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:03.664454 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.664394    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:42:03.742720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.742684    4407 generic.go:356] "Generic (PLEG): container finished" podID="25c58f9d-2706-4af2-8007-252410af7891" containerID="998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542" exitCode=0
Nov 02 03:42:03.742901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.742730    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"
Nov 02 03:42:03.742901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.742770    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" event={"ID":"25c58f9d-2706-4af2-8007-252410af7891","Type":"ContainerDied","Data":"998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542"}
Nov 02 03:42:03.742901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.742821    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w" event={"ID":"25c58f9d-2706-4af2-8007-252410af7891","Type":"ContainerDied","Data":"d6b394ff5a941b234d2650c7fa16ca89d037e3604c6c8f0516bc8fb34a398797"}
Nov 02 03:42:03.742901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.742846    4407 scope.go:122] "RemoveContainer" containerID="998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542"
Nov 02 03:42:03.749755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.749736    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-credential-socket\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.749888 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.749782    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-kube-api-access-tr6h8\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-kube-api-access-tr6h8\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.749888 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.749819    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25c58f9d-2706-4af2-8007-252410af7891-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c58f9d-2706-4af2-8007-252410af7891-istiod-ca-cert\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.749888 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.749883    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-certs\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.750008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.749910    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-envoy\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.750008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.749941    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-data\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.750008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.749998    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-credential-socket" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:03.750121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750005    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-istio-token\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-istio-token\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.750158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750124    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/25c58f9d-2706-4af2-8007-252410af7891-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c58f9d-2706-4af2-8007-252410af7891-istio-podinfo\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.750199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750173    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-socket\") pod \"25c58f9d-2706-4af2-8007-252410af7891\" (UID: \"25c58f9d-2706-4af2-8007-252410af7891\") "
Nov 02 03:42:03.750234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750189    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-certs" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:03.751008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750443    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25c58f9d-2706-4af2-8007-252410af7891-istiod-ca-cert" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:42:03.751008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750900    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:03.751008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750904    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-data" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:03.751008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750923    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:03.751008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750938    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c58f9d-2706-4af2-8007-252410af7891-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:03.751008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.750981    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-socket" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:03.752448 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.752426    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-istio-token" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:42:03.753473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.753443    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-kube-api-access-tr6h8" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "kube-api-access-tr6h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:42:03.753592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.753498    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-envoy" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:03.753592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.753483    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25c58f9d-2706-4af2-8007-252410af7891-istio-podinfo" pod "25c58f9d-2706-4af2-8007-252410af7891" (UID: "25c58f9d-2706-4af2-8007-252410af7891"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:42:03.754156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.754139    4407 scope.go:122] "RemoveContainer" containerID="90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80"
Nov 02 03:42:03.777545 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.777524    4407 scope.go:122] "RemoveContainer" containerID="998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542"
Nov 02 03:42:03.777848 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:42:03.777828    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542\": container with ID starting with 998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542 not found: ID does not exist" containerID="998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542"
Nov 02 03:42:03.777910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.777857    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542"} err="failed to get container status \"998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542\": rpc error: code = NotFound desc = could not find container \"998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542\": container with ID starting with 998d668b0b953aa5be47b33ab459f4e225f67349c570477b6e268d51b47ea542 not found: ID does not exist"
Nov 02 03:42:03.777910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.777874    4407 scope.go:122] "RemoveContainer" containerID="90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80"
Nov 02 03:42:03.778132 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:42:03.778112    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80\": container with ID starting with 90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80 not found: ID does not exist" containerID="90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80"
Nov 02 03:42:03.778182 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.778139    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80"} err="failed to get container status \"90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80\": rpc error: code = NotFound desc = could not find container \"90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80\": container with ID starting with 90cd2871f67a20e30b7dd1cf334737e8aab5baf63447c15887bde085af364d80 not found: ID does not exist"
Nov 02 03:42:03.851939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.851910    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tr6h8\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-kube-api-access-tr6h8\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:03.851939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.851934    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:03.851939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.851946    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:03.852168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.851955    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c58f9d-2706-4af2-8007-252410af7891-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:03.852168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.851963    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c58f9d-2706-4af2-8007-252410af7891-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:03.852168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:03.851971    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c58f9d-2706-4af2-8007-252410af7891-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:04.163706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:04.163674    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"]
Nov 02 03:42:04.170069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:04.170040    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-cb56e9-openshift-default-5985fc47d9-gcs6w"]
Nov 02 03:42:04.351193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:04.351115    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25c58f9d-2706-4af2-8007-252410af7891" path="/var/lib/kubelet/pods/25c58f9d-2706-4af2-8007-252410af7891/volumes"
Nov 02 03:42:05.189324 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:42:05.189285    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:42:06.536885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:06.536847    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:11.278983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:11.278950    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:16.104852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:16.104818    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:20.877398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:20.877318    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:24.918559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:24.918526    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:25.806203 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:25.806165    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:25.962601 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:25.962572    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:26.115205 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:26.115126    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:30.975392 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:30.975365    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:31.221658 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:31.221606    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:42:35.189604 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:42:35.189565    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:42:36.205170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:36.205138    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:38.649262 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:38.649229    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:38.812971 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:38.812931    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:40.730866 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:40.730834    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:42.355922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:42.355888    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:42.577814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:42.577784    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:42.741782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:42.741701    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:43.225252 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:43.225223    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:46.075034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.075004    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"]
Nov 02 03:42:46.075662 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.075647    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy"
Nov 02 03:42:46.075714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.075663    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy"
Nov 02 03:42:46.075714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.075688    4407 cpu_manager.go:414] "containerMap: removing container" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy"
Nov 02 03:42:46.075714 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.075697    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy"
Nov 02 03:42:46.075810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.075804    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy"
Nov 02 03:42:46.075845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.075819    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy"
Nov 02 03:42:46.075845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.075827    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a453c7c-8a6a-4171-8d2a-08217c1d8f7a" containerName="istio-proxy"
Nov 02 03:42:46.079142 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.079127    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104181 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104151    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104194    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwdp\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-kube-api-access-cpwdp\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104387 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104315    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104387 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104343    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104387 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104360    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104450    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104541    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104579    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.104694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.104679    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.132521 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.132495    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-5ac93f-openshift-default-dockercfg-cjnld\""
Nov 02 03:42:46.187675 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.187616    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"]
Nov 02 03:42:46.205342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205313    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.205503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205356    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.205503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205398    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.205503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205422    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.205503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205447    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwdp\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-kube-api-access-cpwdp\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.205752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205537    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.205752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205560    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.205752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205664    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.205752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.205712    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.206101 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.206070    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.206232 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.206177    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.206329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.206307    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.206441 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.206362    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.206441 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.206309    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.207959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.207938    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.208143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.208127    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.263609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.263552    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.264686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.264661    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwdp\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-kube-api-access-cpwdp\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.403389 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.403304    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:46.831471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.831445    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"]
Nov 02 03:42:46.832228 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:42:46.832205    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae2f898_f465_4757_bb66_7e0d51a7b06f.slice/crio-a4bd18e9ceba944413c5b007483dec63134414ce7a2e955594e57edb3eae63da WatchSource:0}: Error finding container a4bd18e9ceba944413c5b007483dec63134414ce7a2e955594e57edb3eae63da: Status 404 returned error can't find the container with id a4bd18e9ceba944413c5b007483dec63134414ce7a2e955594e57edb3eae63da
Nov 02 03:42:46.834796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.834761    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:42:46.834873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.834832    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:42:46.834873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.834861    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:42:46.921685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:46.921648    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:47.010278 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:47.010239    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" event={"ID":"2ae2f898-f465-4757-bb66-7e0d51a7b06f","Type":"ContainerStarted","Data":"d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08"}
Nov 02 03:42:47.010465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:47.010306    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" event={"ID":"2ae2f898-f465-4757-bb66-7e0d51a7b06f","Type":"ContainerStarted","Data":"a4bd18e9ceba944413c5b007483dec63134414ce7a2e955594e57edb3eae63da"}
Nov 02 03:42:47.133089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:47.132979    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podStartSLOduration=1.13296612 podStartE2EDuration="1.13296612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:42:47.13011826 +0000 UTC m=+3310.925466837" watchObservedRunningTime="2026-11-02 03:42:47.13296612 +0000 UTC m=+3310.928314689"
Nov 02 03:42:47.405951 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:47.405858    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:47.408598 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:47.408565    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 03:42:47.408770 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:47.408659    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 03:42:47.666349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:47.666266    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:48.405022 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:48.404986    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 03:42:48.405686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:48.405650    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 03:42:49.000513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:49.000478    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:49.165109 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:49.165074    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:49.273423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:49.273345    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"]
Nov 02 03:42:49.273597 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:49.273543    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy" containerID="cri-o://1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568" gracePeriod=30
Nov 02 03:42:49.342015 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:49.341977    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:49.404928 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:49.404891    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 03:42:49.405257 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:49.405217    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 03:42:50.405089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:50.405050    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 03:42:50.405376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:50.405345    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 03:42:51.406019 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:51.405980    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 03:42:51.406422 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:51.406036    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 03:42:52.406042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:52.406009    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 03:42:52.406560 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:52.406064    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 03:42:52.579208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:52.579177    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:42:53.403868 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:53.403833    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 03:42:53.404045 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:53.403901    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 03:42:54.408341 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.408311    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:54.408839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.408821    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:54.409659 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.409641    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:42:54.553409 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.553354    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:42:54.628509 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628477    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-istio-token\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-istio-token\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.628696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628567    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-kube-api-access-gcw9b\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-kube-api-access-gcw9b\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.628696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628606    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-certs\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.628788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628694    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-credential-socket\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.628788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628722    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/021d20bc-9467-46c6-8705-7b336047a7fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021d20bc-9467-46c6-8705-7b336047a7fa-istio-podinfo\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.628885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628824    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-envoy\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.628970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628948    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/021d20bc-9467-46c6-8705-7b336047a7fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021d20bc-9467-46c6-8705-7b336047a7fa-istiod-ca-cert\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.629030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628993    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-certs" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:54.629030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.629003    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-socket\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.629030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.628988    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-credential-socket" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:54.629190 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.629034    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-data\") pod \"021d20bc-9467-46c6-8705-7b336047a7fa\" (UID: \"021d20bc-9467-46c6-8705-7b336047a7fa\") "
Nov 02 03:42:54.629471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.629313    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021d20bc-9467-46c6-8705-7b336047a7fa-istiod-ca-cert" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:42:54.629471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.629383    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-socket" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:54.629798 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.629744    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-data" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:54.630024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.630001    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021d20bc-9467-46c6-8705-7b336047a7fa-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:54.630107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.630032    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:54.630107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.630049    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:54.630107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.630064    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:54.630107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.630079    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:54.631041 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.631015    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-kube-api-access-gcw9b" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "kube-api-access-gcw9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:42:54.631170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.631151    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-istio-token" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:42:54.631235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.631177    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-envoy" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:42:54.631235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.631176    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/021d20bc-9467-46c6-8705-7b336047a7fa-istio-podinfo" pod "021d20bc-9467-46c6-8705-7b336047a7fa" (UID: "021d20bc-9467-46c6-8705-7b336047a7fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:42:54.731247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.731205    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:54.731247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.731237    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcw9b\" (UniqueName: \"kubernetes.io/projected/021d20bc-9467-46c6-8705-7b336047a7fa-kube-api-access-gcw9b\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:54.731247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.731247    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021d20bc-9467-46c6-8705-7b336047a7fa-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:54.731247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:54.731256    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021d20bc-9467-46c6-8705-7b336047a7fa-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:42:55.068869 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.068836    4407 generic.go:356] "Generic (PLEG): container finished" podID="021d20bc-9467-46c6-8705-7b336047a7fa" containerID="1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568" exitCode=0
Nov 02 03:42:55.069079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.068909    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"
Nov 02 03:42:55.069079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.068921    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" event={"ID":"021d20bc-9467-46c6-8705-7b336047a7fa","Type":"ContainerDied","Data":"1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568"}
Nov 02 03:42:55.069079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.069011    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj" event={"ID":"021d20bc-9467-46c6-8705-7b336047a7fa","Type":"ContainerDied","Data":"43936d8e062c698b473e63cb88341b36c70ff2a38b54c0f7053c8924251eb123"}
Nov 02 03:42:55.069079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.069040    4407 scope.go:122] "RemoveContainer" containerID="1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568"
Nov 02 03:42:55.079678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.079655    4407 scope.go:122] "RemoveContainer" containerID="5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791"
Nov 02 03:42:55.088548 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.088532    4407 scope.go:122] "RemoveContainer" containerID="1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568"
Nov 02 03:42:55.088852 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:42:55.088833    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568\": container with ID starting with 1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568 not found: ID does not exist" containerID="1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568"
Nov 02 03:42:55.088916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.088862    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568"} err="failed to get container status \"1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568\": rpc error: code = NotFound desc = could not find container \"1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568\": container with ID starting with 1473b5669c4e9b53a9ad14095fd93bf1c85d4d2b509ef32269f61520c9714568 not found: ID does not exist"
Nov 02 03:42:55.088916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.088880    4407 scope.go:122] "RemoveContainer" containerID="5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791"
Nov 02 03:42:55.089116 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:42:55.089097    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791\": container with ID starting with 5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791 not found: ID does not exist" containerID="5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791"
Nov 02 03:42:55.089173 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.089126    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791"} err="failed to get container status \"5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791\": rpc error: code = NotFound desc = could not find container \"5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791\": container with ID starting with 5ac35dca6f7f192a1622ac587a02205b39fd2693b7a9ceb8d58caf4861ceb791 not found: ID does not exist"
Nov 02 03:42:55.239172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.239123    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"]
Nov 02 03:42:55.289652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:55.289608    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-3d563c-openshift-default-d89cc6645-zt9kj"]
Nov 02 03:42:56.353334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:56.353297    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" path="/var/lib/kubelet/pods/021d20bc-9467-46c6-8705-7b336047a7fa/volumes"
Nov 02 03:42:57.384042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:42:57.384000    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:02.259851 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:02.259818    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:03.639828 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:03.639801    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:05.188785 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:43:05.188752    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:43:07.660229 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:07.660198    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:08.364506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.364466    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"]
Nov 02 03:43:08.365055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.365039    4407 cpu_manager.go:414] "containerMap: removing container" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy"
Nov 02 03:43:08.365055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.365055    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy"
Nov 02 03:43:08.365197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.365074    4407 cpu_manager.go:414] "containerMap: removing container" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy"
Nov 02 03:43:08.365197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.365079    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25c58f9d-2706-4af2-8007-252410af7891" containerName="istio-proxy"
Nov 02 03:43:08.365197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.365188    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy"
Nov 02 03:43:08.365365 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.365204    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy"
Nov 02 03:43:08.369919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.369899    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.405924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.405889    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-token\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.406095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.405936    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-data\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.406095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.406002    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9bc24-1360-42ed-992c-a28a22efc040-istiod-ca-cert\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.406095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.406034    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-credential-socket\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.406095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.406059    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-envoy\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.406095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.406087    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj6p9\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-kube-api-access-cj6p9\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.406373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.406111    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-certs\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.406373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.406159    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-podinfo\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.406373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.406227    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-socket\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.423030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.422993    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-f68c7e-openshift-default-dockercfg-9xx98\""
Nov 02 03:43:08.478393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.478358    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"]
Nov 02 03:43:08.507556 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.507521    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9bc24-1360-42ed-992c-a28a22efc040-istiod-ca-cert\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.507748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.507571    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-credential-socket\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.507748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.507597    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-envoy\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.507748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.507645    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj6p9\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-kube-api-access-cj6p9\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.507748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.507677    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-certs\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.507748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.507710    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-podinfo\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.508040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.507781    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-socket\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.508040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.507982    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-token\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.508040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.508010    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-data\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.508225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.508008    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-credential-socket\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.508386 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.508358    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9bc24-1360-42ed-992c-a28a22efc040-istiod-ca-cert\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.508599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.508566    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-socket\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.508745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.508600    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-data\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.508745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.508695    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-certs\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.510434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.510411    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-envoy\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.510539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.510446    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-podinfo\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.564979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.564945    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-token\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.565126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.565013    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj6p9\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-kube-api-access-cj6p9\") pod \"e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") " pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:08.682938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:08.682842    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:09.115036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:09.115011    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"]
Nov 02 03:43:09.115403 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:43:09.115367    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a9bc24_1360_42ed_992c_a28a22efc040.slice/crio-0844cd7080a51e3fc20b319bf2194e0e7da6e11ab78e35e1fe41ab8c6e438ff4 WatchSource:0}: Error finding container 0844cd7080a51e3fc20b319bf2194e0e7da6e11ab78e35e1fe41ab8c6e438ff4: Status 404 returned error can't find the container with id 0844cd7080a51e3fc20b319bf2194e0e7da6e11ab78e35e1fe41ab8c6e438ff4
Nov 02 03:43:09.117922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:09.117887    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:43:09.118007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:09.117955    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:43:09.118007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:09.117984    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:43:09.192816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:09.192784    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" event={"ID":"f0a9bc24-1360-42ed-992c-a28a22efc040","Type":"ContainerStarted","Data":"cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12"}
Nov 02 03:43:09.192972 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:09.192847    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" event={"ID":"f0a9bc24-1360-42ed-992c-a28a22efc040","Type":"ContainerStarted","Data":"0844cd7080a51e3fc20b319bf2194e0e7da6e11ab78e35e1fe41ab8c6e438ff4"}
Nov 02 03:43:09.313398 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:09.313351    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" podStartSLOduration=1.313338536 podStartE2EDuration="1.313338536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:43:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:43:09.312297081 +0000 UTC m=+3333.107645646" watchObservedRunningTime="2026-11-02 03:43:09.313338536 +0000 UTC m=+3333.108687101"
Nov 02 03:43:09.705800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:09.705747    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:10.408222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:10.408176    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:10.693791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:10.693705    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Nov 02 03:43:10.694021 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:10.693993    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Nov 02 03:43:11.686651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:11.685988    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Nov 02 03:43:11.686651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:11.686050    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Nov 02 03:43:12.239773 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:12.239733    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:12.701190 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:12.701144    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Nov 02 03:43:12.705767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:12.705730    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Nov 02 03:43:13.705652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:13.691010    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Nov 02 03:43:13.705652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:13.691069    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Nov 02 03:43:13.948001 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:13.947970    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:14.240790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:14.240737    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:14.405704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:14.404069    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:14.697024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:14.696987    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Nov 02 03:43:14.697312 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:14.697284    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Nov 02 03:43:14.845853 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:14.845802    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:15.688175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:15.688150    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:15.688651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:15.688633    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:15.689273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:15.689259    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:43:19.572821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:19.572783    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:20.627420 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:20.627378    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:20.785007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:20.784977    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:21.091406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:21.091374    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:24.826653 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:24.826609    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:29.762313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:29.762280    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:30.603816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:30.603783    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:43:34.583294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:34.583242    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:35.189241 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:43:35.189209    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:43:39.006845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:39.006811    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:39.156904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:39.156872    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:39.376704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:39.376609    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:44.372361 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:44.372325    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:44.774354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:44.774322    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:47.333929 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:47.333898    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:49.173785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:49.173664    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:51.976593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:51.976563    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:53.425555 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:53.425517    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:54.940115 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:54.940070    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:55.179251 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:55.179222    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:55.338001 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:55.337966    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:43:55.830957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:43:55.830903    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:00.198687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:00.198653    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:01.985075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:01.985036    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:02.150260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:02.150225    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:02.283833 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:02.283807    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"]
Nov 02 03:44:02.284057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:02.284027    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy" containerID="cri-o://b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245" gracePeriod=30
Nov 02 03:44:02.321353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:02.321322    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:05.074936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:05.074902    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:05.188235 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:44:05.188202    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:44:07.544481 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.544454    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:44:07.561729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.561635    4407 generic.go:356] "Generic (PLEG): container finished" podID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerID="b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245" exitCode=0
Nov 02 03:44:07.561729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.561665    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" event={"ID":"049b989c-9dbc-4cc0-8177-a3d83778296f","Type":"ContainerDied","Data":"b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245"}
Nov 02 03:44:07.561729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.561704    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"
Nov 02 03:44:07.561958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.561777    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb" event={"ID":"049b989c-9dbc-4cc0-8177-a3d83778296f","Type":"ContainerDied","Data":"7966085c5c82f0f775048b5bf3a8345f43b48b0b7bc08dcbdc59780cbb1d0ee3"}
Nov 02 03:44:07.561958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.561810    4407 scope.go:122] "RemoveContainer" containerID="b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245"
Nov 02 03:44:07.573092 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.573070    4407 scope.go:122] "RemoveContainer" containerID="cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12"
Nov 02 03:44:07.583589 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.583565    4407 scope.go:122] "RemoveContainer" containerID="b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245"
Nov 02 03:44:07.583972 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:44:07.583952    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245\": container with ID starting with b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245 not found: ID does not exist" containerID="b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245"
Nov 02 03:44:07.584086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.583985    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245"} err="failed to get container status \"b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245\": rpc error: code = NotFound desc = could not find container \"b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245\": container with ID starting with b283068e8e509b90de607cd8a06733b3a7eb5bef704795854143c76b90e7b245 not found: ID does not exist"
Nov 02 03:44:07.584086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.584009    4407 scope.go:122] "RemoveContainer" containerID="cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12"
Nov 02 03:44:07.584362 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:44:07.584339    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12\": container with ID starting with cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12 not found: ID does not exist" containerID="cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12"
Nov 02 03:44:07.584421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.584371    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12"} err="failed to get container status \"cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12\": rpc error: code = NotFound desc = could not find container \"cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12\": container with ID starting with cb5a7a8d5f08dab32265a5f089ce7642195c2639791e747fc319539a1ea3ee12 not found: ID does not exist"
Nov 02 03:44:07.629203 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629169    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-podinfo\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629215    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-envoy\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629250    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-kube-api-access-fbbxb\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-kube-api-access-fbbxb\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629277    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-certs\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629305    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-credential-socket\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629330    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-token\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-token\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629367    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-socket\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629440    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/049b989c-9dbc-4cc0-8177-a3d83778296f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/049b989c-9dbc-4cc0-8177-a3d83778296f-istiod-ca-cert\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629475    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-data\") pod \"049b989c-9dbc-4cc0-8177-a3d83778296f\" (UID: \"049b989c-9dbc-4cc0-8177-a3d83778296f\") "
Nov 02 03:44:07.629755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629618    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-certs" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:07.629755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629665    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-credential-socket" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:07.629953 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629805    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-socket" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:07.629953 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629928    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-data" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:07.630047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.629981    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/049b989c-9dbc-4cc0-8177-a3d83778296f-istiod-ca-cert" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:44:07.630114 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.630100    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.630157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.630118    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.630157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.630131    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.630157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.630143    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.630157 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.630151    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/049b989c-9dbc-4cc0-8177-a3d83778296f-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.632259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.632233    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-podinfo" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:44:07.632383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.632259    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-envoy" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:07.632383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.632296    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-token" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:44:07.632383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.632322    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-kube-api-access-fbbxb" pod "049b989c-9dbc-4cc0-8177-a3d83778296f" (UID: "049b989c-9dbc-4cc0-8177-a3d83778296f"). InnerVolumeSpecName "kube-api-access-fbbxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:44:07.731640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.731600    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.731640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.731645    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.731846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.731656    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbbxb\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-kube-api-access-fbbxb\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.731846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.731665    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/049b989c-9dbc-4cc0-8177-a3d83778296f-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:07.987195 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.987161    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"]
Nov 02 03:44:07.989942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:07.989918    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-2912e6-openshift-default-77c65cbbfc-bz4mb"]
Nov 02 03:44:08.351796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:08.351764    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" path="/var/lib/kubelet/pods/049b989c-9dbc-4cc0-8177-a3d83778296f/volumes"
Nov 02 03:44:10.160457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:10.160413    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:15.472787 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:15.472751    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:19.073117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:19.073085    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"]
Nov 02 03:44:19.179136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:19.179102    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"]
Nov 02 03:44:19.461742 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:19.461656    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:19.619848 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:19.619815    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:20.657927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:20.657899    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:24.074141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.074055    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq" podUID="f6269d59-e3bc-4fe0-acc9-02bbaeefd570" containerName="payload-processing" containerID="cri-o://b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563" gracePeriod=25
Nov 02 03:44:24.077238 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.077214    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:44:24.180322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.180250    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch" podUID="c2230c72-35df-4e94-8759-273f34800d71" containerName="payload-pre-processing" containerID="cri-o://1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e" gracePeriod=25
Nov 02 03:44:24.363894 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.363870    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:44:24.453383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.450441    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-plugins-config-volume\") pod \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\" (UID: \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\") "
Nov 02 03:44:24.453383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.450567    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-kube-api-access-wwh9g\" (UniqueName: \"kubernetes.io/projected/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-kube-api-access-wwh9g\") pod \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\" (UID: \"f6269d59-e3bc-4fe0-acc9-02bbaeefd570\") "
Nov 02 03:44:24.453383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.451978    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-plugins-config-volume" pod "f6269d59-e3bc-4fe0-acc9-02bbaeefd570" (UID: "f6269d59-e3bc-4fe0-acc9-02bbaeefd570"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:44:24.477181 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.477124    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-kube-api-access-wwh9g" pod "f6269d59-e3bc-4fe0-acc9-02bbaeefd570" (UID: "f6269d59-e3bc-4fe0-acc9-02bbaeefd570"). InnerVolumeSpecName "kube-api-access-wwh9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:44:24.520813 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.520785    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:44:24.552559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.552526    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:24.552559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.552551    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwh9g\" (UniqueName: \"kubernetes.io/projected/f6269d59-e3bc-4fe0-acc9-02bbaeefd570-kube-api-access-wwh9g\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:24.657725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.655611    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2230c72-35df-4e94-8759-273f34800d71-kube-api-access-5qnkn\" (UniqueName: \"kubernetes.io/projected/c2230c72-35df-4e94-8759-273f34800d71-kube-api-access-5qnkn\") pod \"c2230c72-35df-4e94-8759-273f34800d71\" (UID: \"c2230c72-35df-4e94-8759-273f34800d71\") "
Nov 02 03:44:24.657725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.655897    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2230c72-35df-4e94-8759-273f34800d71-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2230c72-35df-4e94-8759-273f34800d71-plugins-config-volume\") pod \"c2230c72-35df-4e94-8759-273f34800d71\" (UID: \"c2230c72-35df-4e94-8759-273f34800d71\") "
Nov 02 03:44:24.657725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.656770    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2230c72-35df-4e94-8759-273f34800d71-plugins-config-volume" pod "c2230c72-35df-4e94-8759-273f34800d71" (UID: "c2230c72-35df-4e94-8759-273f34800d71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:44:24.659453 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.659419    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2230c72-35df-4e94-8759-273f34800d71-kube-api-access-5qnkn" pod "c2230c72-35df-4e94-8759-273f34800d71" (UID: "c2230c72-35df-4e94-8759-273f34800d71"). InnerVolumeSpecName "kube-api-access-5qnkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:44:24.671904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.671869    4407 generic.go:356] "Generic (PLEG): container finished" podID="f6269d59-e3bc-4fe0-acc9-02bbaeefd570" containerID="b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563" exitCode=0
Nov 02 03:44:24.672118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.671928    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"
Nov 02 03:44:24.672118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.671997    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq" event={"ID":"f6269d59-e3bc-4fe0-acc9-02bbaeefd570","Type":"ContainerDied","Data":"b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563"}
Nov 02 03:44:24.672118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.672040    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq" event={"ID":"f6269d59-e3bc-4fe0-acc9-02bbaeefd570","Type":"ContainerDied","Data":"eac548c011991f4e9f9debbf31b20f67dc0bd1c08dfe07e82647cccb5cd64471"}
Nov 02 03:44:24.672118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.672064    4407 scope.go:122] "RemoveContainer" containerID="b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563"
Nov 02 03:44:24.683830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.681143    4407 generic.go:356] "Generic (PLEG): container finished" podID="c2230c72-35df-4e94-8759-273f34800d71" containerID="1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e" exitCode=0
Nov 02 03:44:24.683830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.681286    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch" event={"ID":"c2230c72-35df-4e94-8759-273f34800d71","Type":"ContainerDied","Data":"1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e"}
Nov 02 03:44:24.683830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.681333    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch" event={"ID":"c2230c72-35df-4e94-8759-273f34800d71","Type":"ContainerDied","Data":"63db99b19b90d112c48a8c9008e4e84bab1db3d484d1b710f5ee02158b5bad52"}
Nov 02 03:44:24.683830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.681408    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"
Nov 02 03:44:24.694771 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.694750    4407 scope.go:122] "RemoveContainer" containerID="b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563"
Nov 02 03:44:24.699984 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:44:24.699948    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563\": container with ID starting with b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563 not found: ID does not exist" containerID="b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563"
Nov 02 03:44:24.700113 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.699995    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563"} err="failed to get container status \"b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563\": rpc error: code = NotFound desc = could not find container \"b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563\": container with ID starting with b0c9f7b30814e06e815592efa627c2113c11271f9fb71f996ba1ba180741f563 not found: ID does not exist"
Nov 02 03:44:24.700113 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.700017    4407 scope.go:122] "RemoveContainer" containerID="1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e"
Nov 02 03:44:24.713037 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.713012    4407 scope.go:122] "RemoveContainer" containerID="1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e"
Nov 02 03:44:24.713367 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:44:24.713330    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e\": container with ID starting with 1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e not found: ID does not exist" containerID="1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e"
Nov 02 03:44:24.713463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.713374    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e"} err="failed to get container status \"1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e\": rpc error: code = NotFound desc = could not find container \"1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e\": container with ID starting with 1c40a992a44d8cca5ebce6e6327fd50ff2fddf58eb8b835df79cf694a2340c6e not found: ID does not exist"
Nov 02 03:44:24.758065 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.758031    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2230c72-35df-4e94-8759-273f34800d71-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:24.758065 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.758066    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qnkn\" (UniqueName: \"kubernetes.io/projected/c2230c72-35df-4e94-8759-273f34800d71-kube-api-access-5qnkn\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:24.841121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.841085    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"]
Nov 02 03:44:24.846718 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.846690    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-ad96f4-755fc48bc5-24zch"]
Nov 02 03:44:24.959483 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.959395    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"]
Nov 02 03:44:24.964197 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.964171    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-ad96f4-5679dbf797-tt6bq"]
Nov 02 03:44:24.976501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:24.976473    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:26.355705 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:26.355674    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2230c72-35df-4e94-8759-273f34800d71" path="/var/lib/kubelet/pods/c2230c72-35df-4e94-8759-273f34800d71/volumes"
Nov 02 03:44:26.356108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:26.356094    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6269d59-e3bc-4fe0-acc9-02bbaeefd570" path="/var/lib/kubelet/pods/f6269d59-e3bc-4fe0-acc9-02bbaeefd570/volumes"
Nov 02 03:44:26.671013 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:26.670935    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:26.859648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:26.859597    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:27.014042 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:27.014013    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:27.173629 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:27.173583    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:27.515506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:27.515478    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:31.911150 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:31.911117    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:33.351577 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:33.351546    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:33.517802 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:33.517765    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:33.648056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:33.647980    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"]
Nov 02 03:44:33.648185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:33.648155    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy" containerID="cri-o://cdebd7263ac6dcd02dcca661bf14c6e02b782909a7d0069385feae000b5d7449" gracePeriod=30
Nov 02 03:44:33.677269 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:33.677238    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:35.188486 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:44:35.188446    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:44:36.679494 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:36.679463    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:38.787226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:38.787194    4407 generic.go:356] "Generic (PLEG): container finished" podID="d1946862-d724-4984-a6a7-2a44608f1658" containerID="cdebd7263ac6dcd02dcca661bf14c6e02b782909a7d0069385feae000b5d7449" exitCode=0
Nov 02 03:44:38.787613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:38.787257    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" event={"ID":"d1946862-d724-4984-a6a7-2a44608f1658","Type":"ContainerDied","Data":"cdebd7263ac6dcd02dcca661bf14c6e02b782909a7d0069385feae000b5d7449"}
Nov 02 03:44:38.787613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:38.787354    4407 scope.go:122] "RemoveContainer" containerID="2a6b8d39282b751712dd4b7e1cef33084871fc4b5e2b8420822ec08b71cfdf63"
Nov 02 03:44:38.939107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:38.939037    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:44:39.046743 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.046706    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-credential-socket\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.046922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.046813    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-istio-token\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-istio-token\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.046922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.046845    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-certs\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.046922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.046878    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-socket\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.047057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.046931    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-envoy\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.047057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047002    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1946862-d724-4984-a6a7-2a44608f1658-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1946862-d724-4984-a6a7-2a44608f1658-istio-podinfo\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.047057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047028    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-data\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.047057 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047053    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1946862-d724-4984-a6a7-2a44608f1658-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1946862-d724-4984-a6a7-2a44608f1658-istiod-ca-cert\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.047276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047049    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-credential-socket" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:39.047276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047139    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-kube-api-access-9s5j9\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-kube-api-access-9s5j9\") pod \"d1946862-d724-4984-a6a7-2a44608f1658\" (UID: \"d1946862-d724-4984-a6a7-2a44608f1658\") "
Nov 02 03:44:39.047276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047166    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-certs" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:39.047276 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047253    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-socket" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:39.047507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047354    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-data" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:39.047600 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047579    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1946862-d724-4984-a6a7-2a44608f1658-istiod-ca-cert" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:44:39.047699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047682    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.047747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047708    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.047747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047722    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.047747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.047735    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.049435 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.049410    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-kube-api-access-9s5j9" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "kube-api-access-9s5j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:44:39.049595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.049576    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1946862-d724-4984-a6a7-2a44608f1658-istio-podinfo" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:44:39.049859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.049834    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-istio-token" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:44:39.049859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.049833    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-envoy" pod "d1946862-d724-4984-a6a7-2a44608f1658" (UID: "d1946862-d724-4984-a6a7-2a44608f1658"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:44:39.149320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.149284    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.149320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.149314    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1946862-d724-4984-a6a7-2a44608f1658-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.149320 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.149323    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1946862-d724-4984-a6a7-2a44608f1658-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.149560 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.149332    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1946862-d724-4984-a6a7-2a44608f1658-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.149560 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.149349    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s5j9\" (UniqueName: \"kubernetes.io/projected/d1946862-d724-4984-a6a7-2a44608f1658-kube-api-access-9s5j9\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:44:39.794288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.794259    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"
Nov 02 03:44:39.794288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.794270    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d" event={"ID":"d1946862-d724-4984-a6a7-2a44608f1658","Type":"ContainerDied","Data":"e57dce7d2ff02d908c0dd87fd87bf4567101e4bcc95d61f9c9b9d686b51c586c"}
Nov 02 03:44:39.794800 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.794324    4407 scope.go:122] "RemoveContainer" containerID="cdebd7263ac6dcd02dcca661bf14c6e02b782909a7d0069385feae000b5d7449"
Nov 02 03:44:39.960450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:39.960423    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"]
Nov 02 03:44:40.011324 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:40.011289    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-9b06f8-openshift-default-78cf45cfb9-4rk6d"]
Nov 02 03:44:40.351187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:40.351155    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1946862-d724-4984-a6a7-2a44608f1658" path="/var/lib/kubelet/pods/d1946862-d724-4984-a6a7-2a44608f1658/volumes"
Nov 02 03:44:41.435240 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:41.435212    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:46.280788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:46.280758    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:47.605271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:47.605235    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:51.104676 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:51.104615    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:55.265164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:55.265120    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:56.776969 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:56.776920    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:57.065856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:57.065763    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:57.218122 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:57.218094    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:44:57.666433 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:44:57.666403    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:02.276837 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.276801    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:02.577876 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.577792    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"]
Nov 02 03:45:02.578311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578299    4407 cpu_manager.go:414] "containerMap: removing container" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy"
Nov 02 03:45:02.578358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578312    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy"
Nov 02 03:45:02.578358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578322    4407 cpu_manager.go:414] "containerMap: removing container" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy"
Nov 02 03:45:02.578358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578327    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy"
Nov 02 03:45:02.578358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578338    4407 cpu_manager.go:414] "containerMap: removing container" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy"
Nov 02 03:45:02.578358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578344    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="021d20bc-9467-46c6-8705-7b336047a7fa" containerName="istio-proxy"
Nov 02 03:45:02.578358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578352    4407 cpu_manager.go:414] "containerMap: removing container" podUID="c2230c72-35df-4e94-8759-273f34800d71" containerName="payload-pre-processing"
Nov 02 03:45:02.578358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578357    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2230c72-35df-4e94-8759-273f34800d71" containerName="payload-pre-processing"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578365    4407 cpu_manager.go:414] "containerMap: removing container" podUID="f6269d59-e3bc-4fe0-acc9-02bbaeefd570" containerName="payload-processing"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578369    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6269d59-e3bc-4fe0-acc9-02bbaeefd570" containerName="payload-processing"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578385    4407 cpu_manager.go:414] "containerMap: removing container" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578389    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578470    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578478    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578486    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2230c72-35df-4e94-8759-273f34800d71" containerName="payload-pre-processing"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578491    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6269d59-e3bc-4fe0-acc9-02bbaeefd570" containerName="payload-processing"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578500    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="049b989c-9dbc-4cc0-8177-a3d83778296f" containerName="istio-proxy"
Nov 02 03:45:02.578568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.578508    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy"
Nov 02 03:45:02.583131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.583108    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.637663 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.637612    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"]
Nov 02 03:45:02.642942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.642907    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.643144 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.643085    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dzk2\" (UniqueName: \"kubernetes.io/projected/8b1a180c-29f9-4f05-8719-7af8d044425f-kube-api-access-2dzk2\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.643144 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.643115    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.675747 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.675710    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 03:45:02.684864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.684840    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9z5bk\""
Nov 02 03:45:02.744930 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.744893    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.745136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.745088    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dzk2\" (UniqueName: \"kubernetes.io/projected/8b1a180c-29f9-4f05-8719-7af8d044425f-kube-api-access-2dzk2\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.745136 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.745119    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.745667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.745645    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.747726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.747700    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.803040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.803011    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dzk2\" (UniqueName: \"kubernetes.io/projected/8b1a180c-29f9-4f05-8719-7af8d044425f-kube-api-access-2dzk2\") pod \"maas-api-revoke-sub-91858a07-phnqp\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:02.897017 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:02.896939    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:03.269994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:03.269970    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"]
Nov 02 03:45:03.270693 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:45:03.270668    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1a180c_29f9_4f05_8719_7af8d044425f.slice/crio-d788bade07ec294e0eadc30f09eb496539ef34ddc210b0d4b3b5d4b6703504be WatchSource:0}: Error finding container d788bade07ec294e0eadc30f09eb496539ef34ddc210b0d4b3b5d4b6703504be: Status 404 returned error can't find the container with id d788bade07ec294e0eadc30f09eb496539ef34ddc210b0d4b3b5d4b6703504be
Nov 02 03:45:03.272599 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:03.272584    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 03:45:03.933013 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:03.932979    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp" event={"ID":"8b1a180c-29f9-4f05-8719-7af8d044425f","Type":"ContainerStarted","Data":"d788bade07ec294e0eadc30f09eb496539ef34ddc210b0d4b3b5d4b6703504be"}
Nov 02 03:45:04.031980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:04.031942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:04.209392 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:04.209298    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:04.375578 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:04.375541    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:05.189520 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:45:05.189480    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:45:05.540167 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.540135    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"]
Nov 02 03:45:05.540665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.540651    4407 cpu_manager.go:414] "containerMap: removing container" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy"
Nov 02 03:45:05.540665 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.540665    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1946862-d724-4984-a6a7-2a44608f1658" containerName="istio-proxy"
Nov 02 03:45:05.545244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.545225    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.601296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.601257    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"]
Nov 02 03:45:05.683584 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.683551    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.683740 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.683591    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.683888 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.683866    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk4lc\" (UniqueName: \"kubernetes.io/projected/3ac82e3e-6052-47f0-adce-a5a845b6694b-kube-api-access-lk4lc\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.785166 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.785130    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk4lc\" (UniqueName: \"kubernetes.io/projected/3ac82e3e-6052-47f0-adce-a5a845b6694b-kube-api-access-lk4lc\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.785348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.785196    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.785348 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.785216    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.785823 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.785800    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.787682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.787662    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.845280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.845206    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk4lc\" (UniqueName: \"kubernetes.io/projected/3ac82e3e-6052-47f0-adce-a5a845b6694b-kube-api-access-lk4lc\") pod \"maas-api-revoke-sub-9fd61eb2-6hk8c\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:05.860105 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:05.860080    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:06.236742 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:06.236711    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"]
Nov 02 03:45:06.237178 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:45:06.236754    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac82e3e_6052_47f0_adce_a5a845b6694b.slice/crio-7d3cfe205980b8af643b15f9187ddbe5805be0ed11b72cd032fb26f1b9170f4a WatchSource:0}: Error finding container 7d3cfe205980b8af643b15f9187ddbe5805be0ed11b72cd032fb26f1b9170f4a: Status 404 returned error can't find the container with id 7d3cfe205980b8af643b15f9187ddbe5805be0ed11b72cd032fb26f1b9170f4a
Nov 02 03:45:06.952816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:06.952781    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c" event={"ID":"3ac82e3e-6052-47f0-adce-a5a845b6694b","Type":"ContainerStarted","Data":"7d3cfe205980b8af643b15f9187ddbe5805be0ed11b72cd032fb26f1b9170f4a"}
Nov 02 03:45:07.046047 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:07.046012    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:10.976062 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:10.976026    4407 generic.go:356] "Generic (PLEG): container finished" podID="3ac82e3e-6052-47f0-adce-a5a845b6694b" containerID="907b16f768528f777f621b1bd792c9e08217345ca94160ec414f3340dfdb68d9" exitCode=0
Nov 02 03:45:10.976515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:10.976115    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c" event={"ID":"3ac82e3e-6052-47f0-adce-a5a845b6694b","Type":"ContainerDied","Data":"907b16f768528f777f621b1bd792c9e08217345ca94160ec414f3340dfdb68d9"}
Nov 02 03:45:11.807741 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:11.807710    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:12.118613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.118589    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:12.262682 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.262649    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-openshift-ca\") pod \"3ac82e3e-6052-47f0-adce-a5a845b6694b\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") "
Nov 02 03:45:12.262852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.262772    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-xks-ca\") pod \"3ac82e3e-6052-47f0-adce-a5a845b6694b\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") "
Nov 02 03:45:12.262852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.262844    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ac82e3e-6052-47f0-adce-a5a845b6694b-kube-api-access-lk4lc\" (UniqueName: \"kubernetes.io/projected/3ac82e3e-6052-47f0-adce-a5a845b6694b-kube-api-access-lk4lc\") pod \"3ac82e3e-6052-47f0-adce-a5a845b6694b\" (UID: \"3ac82e3e-6052-47f0-adce-a5a845b6694b\") "
Nov 02 03:45:12.262983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.262964    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-openshift-ca" pod "3ac82e3e-6052-47f0-adce-a5a845b6694b" (UID: "3ac82e3e-6052-47f0-adce-a5a845b6694b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:45:12.263375 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.263355    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-openshift-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:45:12.264994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.264968    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-xks-ca" pod "3ac82e3e-6052-47f0-adce-a5a845b6694b" (UID: "3ac82e3e-6052-47f0-adce-a5a845b6694b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:45:12.265313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.265284    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ac82e3e-6052-47f0-adce-a5a845b6694b-kube-api-access-lk4lc" pod "3ac82e3e-6052-47f0-adce-a5a845b6694b" (UID: "3ac82e3e-6052-47f0-adce-a5a845b6694b"). InnerVolumeSpecName "kube-api-access-lk4lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:45:12.364328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.364301    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ac82e3e-6052-47f0-adce-a5a845b6694b-maas-api-xks-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:45:12.364328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.364328    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk4lc\" (UniqueName: \"kubernetes.io/projected/3ac82e3e-6052-47f0-adce-a5a845b6694b-kube-api-access-lk4lc\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:45:12.988252 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.988222    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"
Nov 02 03:45:12.988419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.988254    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c" event={"ID":"3ac82e3e-6052-47f0-adce-a5a845b6694b","Type":"ContainerDied","Data":"7d3cfe205980b8af643b15f9187ddbe5805be0ed11b72cd032fb26f1b9170f4a"}
Nov 02 03:45:12.988419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:12.988307    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d3cfe205980b8af643b15f9187ddbe5805be0ed11b72cd032fb26f1b9170f4a"
Nov 02 03:45:13.341126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:13.341092    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:45:15.006475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:15.006440    4407 generic.go:356] "Generic (PLEG): container finished" podID="8b1a180c-29f9-4f05-8719-7af8d044425f" containerID="ebab1f20942352026b6500496f09c450beb98b08457b8bb6286ea7061ef453e4" exitCode=0
Nov 02 03:45:15.007002 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:15.006565    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp" event={"ID":"8b1a180c-29f9-4f05-8719-7af8d044425f","Type":"ContainerDied","Data":"ebab1f20942352026b6500496f09c450beb98b08457b8bb6286ea7061ef453e4"}
Nov 02 03:45:16.172314 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.172289    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:16.317794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.317709    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-openshift-ca\") pod \"8b1a180c-29f9-4f05-8719-7af8d044425f\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") "
Nov 02 03:45:16.317794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.317781    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b1a180c-29f9-4f05-8719-7af8d044425f-kube-api-access-2dzk2\" (UniqueName: \"kubernetes.io/projected/8b1a180c-29f9-4f05-8719-7af8d044425f-kube-api-access-2dzk2\") pod \"8b1a180c-29f9-4f05-8719-7af8d044425f\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") "
Nov 02 03:45:16.318040 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.317934    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-xks-ca\") pod \"8b1a180c-29f9-4f05-8719-7af8d044425f\" (UID: \"8b1a180c-29f9-4f05-8719-7af8d044425f\") "
Nov 02 03:45:16.318168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.318133    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-openshift-ca" pod "8b1a180c-29f9-4f05-8719-7af8d044425f" (UID: "8b1a180c-29f9-4f05-8719-7af8d044425f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:45:16.318446 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.318424    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-openshift-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:45:16.320059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.320037    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b1a180c-29f9-4f05-8719-7af8d044425f-kube-api-access-2dzk2" pod "8b1a180c-29f9-4f05-8719-7af8d044425f" (UID: "8b1a180c-29f9-4f05-8719-7af8d044425f"). InnerVolumeSpecName "kube-api-access-2dzk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:45:16.320193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.320172    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-xks-ca" pod "8b1a180c-29f9-4f05-8719-7af8d044425f" (UID: "8b1a180c-29f9-4f05-8719-7af8d044425f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:45:16.419117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.419088    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1a180c-29f9-4f05-8719-7af8d044425f-maas-api-xks-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:45:16.419117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.419117    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dzk2\" (UniqueName: \"kubernetes.io/projected/8b1a180c-29f9-4f05-8719-7af8d044425f-kube-api-access-2dzk2\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:45:16.648579 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:16.648497    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:17.019242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:17.019208    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp" event={"ID":"8b1a180c-29f9-4f05-8719-7af8d044425f","Type":"ContainerDied","Data":"d788bade07ec294e0eadc30f09eb496539ef34ddc210b0d4b3b5d4b6703504be"}
Nov 02 03:45:17.019242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:17.019238    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"
Nov 02 03:45:17.019452 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:17.019257    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d788bade07ec294e0eadc30f09eb496539ef34ddc210b0d4b3b5d4b6703504be"
Nov 02 03:45:21.403009 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:21.402923    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:24.596226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:24.596188    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:25.188124 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:25.188092    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:25.357568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:25.357525    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:26.378432 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:26.378402    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:26.630345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:26.630256    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:26.789790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:26.789762    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:27.101821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.101785    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"]
Nov 02 03:45:27.102416 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.102396    4407 cpu_manager.go:414] "containerMap: removing container" podUID="3ac82e3e-6052-47f0-adce-a5a845b6694b" containerName="revoke-keys"
Nov 02 03:45:27.102524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.102418    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ac82e3e-6052-47f0-adce-a5a845b6694b" containerName="revoke-keys"
Nov 02 03:45:27.102524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.102501    4407 cpu_manager.go:414] "containerMap: removing container" podUID="8b1a180c-29f9-4f05-8719-7af8d044425f" containerName="revoke-keys"
Nov 02 03:45:27.102524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.102510    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b1a180c-29f9-4f05-8719-7af8d044425f" containerName="revoke-keys"
Nov 02 03:45:27.102731 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.102658    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b1a180c-29f9-4f05-8719-7af8d044425f" containerName="revoke-keys"
Nov 02 03:45:27.102731 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.102680    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ac82e3e-6052-47f0-adce-a5a845b6694b" containerName="revoke-keys"
Nov 02 03:45:27.106242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.106223    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.189742 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.189704    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-podinfo\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.190010 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.189990    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tck2t\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-kube-api-access-tck2t\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.190147 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.190131    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac08935-e676-4ffc-ab77-4abc80198b6c-istiod-ca-cert\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.190267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.190253    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-data\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.190404 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.190390    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-credential-socket\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.190540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.190526    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-envoy\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.190678 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.190660    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-socket\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.190807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.190793    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-certs\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.191086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.191048    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-token\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.193568 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.193547    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"]
Nov 02 03:45:27.214339 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.214303    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:27.228597 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.228566    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-a7cf93-openshift-default-dockercfg-h8s77\""
Nov 02 03:45:27.292483 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292455    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-token\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.292692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292522    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-podinfo\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.292692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292548    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tck2t\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-kube-api-access-tck2t\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.292692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292583    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac08935-e676-4ffc-ab77-4abc80198b6c-istiod-ca-cert\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.292692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292612    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-data\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.292692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292685    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-credential-socket\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.292965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292741    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-envoy\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.292965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292865    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-socket\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.292965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.292921    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-certs\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.293174 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.293146    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-data\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.293275 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.293214    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-credential-socket\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.293318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.293278    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-certs\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.293370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.293315    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-socket\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.293434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.293418    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac08935-e676-4ffc-ab77-4abc80198b6c-istiod-ca-cert\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.295260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.295237    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-envoy\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.295357 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.295344    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-podinfo\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.352831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.352762    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-token\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.352831 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.352807    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tck2t\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-kube-api-access-tck2t\") pod \"e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") " pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.420222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.420189    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:27.428527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.428489    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:27.602752 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.599448    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:27.787010 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.786979    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:27.913049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.913025    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"]
Nov 02 03:45:27.914173 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:45:27.914133    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac08935_e676_4ffc_ab77_4abc80198b6c.slice/crio-42861ce3c531ac1f80bd52f8ac71da8fec9296e382c7e68fbcc76400ba45c46a WatchSource:0}: Error finding container 42861ce3c531ac1f80bd52f8ac71da8fec9296e382c7e68fbcc76400ba45c46a: Status 404 returned error can't find the container with id 42861ce3c531ac1f80bd52f8ac71da8fec9296e382c7e68fbcc76400ba45c46a
Nov 02 03:45:27.916724 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.916691    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:27.916814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.916758    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:27.916814 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:27.916792    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:28.083487 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:28.083403    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" event={"ID":"6ac08935-e676-4ffc-ab77-4abc80198b6c","Type":"ContainerStarted","Data":"1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc"}
Nov 02 03:45:28.083487 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:28.083464    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" event={"ID":"6ac08935-e676-4ffc-ab77-4abc80198b6c","Type":"ContainerStarted","Data":"42861ce3c531ac1f80bd52f8ac71da8fec9296e382c7e68fbcc76400ba45c46a"}
Nov 02 03:45:28.201465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:28.201417    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podStartSLOduration=1.201401886 podStartE2EDuration="1.201401886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:45:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:45:28.200369865 +0000 UTC m=+3471.995718432" watchObservedRunningTime="2026-11-02 03:45:28.201401886 +0000 UTC m=+3471.996750450"
Nov 02 03:45:28.420815 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:28.420727    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:28.426480 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:28.426458    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:29.088563 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:29.088531    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:29.089648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:29.089602    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:45:29.107169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:29.107142    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:29.295313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:29.295284    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:29.462107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:29.461942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:31.610512 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:31.610479    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:31.972058 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:31.971962    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:32.154539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:32.154507    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:32.319527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:32.319498    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:32.482652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:32.482606    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:32.717127 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:32.717052    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:33.261979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:33.261942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:33.442841 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:33.442810    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:33.622148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:33.622057    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:35.189459 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:45:35.189426    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:45:36.906684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:36.906656    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:37.169860 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:37.169756    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:37.325638 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:37.325586    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:39.714117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:39.714085    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:39.992308 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:39.992226    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:40.157656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:40.157596    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:42.289368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:42.289335    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:44.179602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:44.179532    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:44.352145 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:44.352108    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:47.443612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.443577    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"]
Nov 02 03:45:47.452230 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.452203    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.477942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.477890    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:47.510507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.510463    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"]
Nov 02 03:45:47.515249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.515225    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.549906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.549876    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"]
Nov 02 03:45:47.600228 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.600186    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"]
Nov 02 03:45:47.600603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.600581    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.600704 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.600636    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.600819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.600800    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpwz4\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-kube-api-access-bpwz4\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.600898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.600873    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05282817-c06b-4790-bb94-cb414c255936-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.600986 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.600967    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05282817-c06b-4790-bb94-cb414c255936-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.601052 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.601011    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.601152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.601126    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.601211 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.601190    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.601380 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.601360    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.702766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.702680    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.702766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.702726    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrvzv\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-kube-api-access-rrvzv\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.702766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.702759    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.703059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.702779    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.703059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.702809    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.703059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.702960    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.702978    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.703059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.702999    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.703059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703016    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.703059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703039    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703153    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703204    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.703421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703265    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.703421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703358    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-credential-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703374    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-certs\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703375    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpwz4\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-kube-api-access-bpwz4\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703421 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703415    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05282817-c06b-4790-bb94-cb414c255936-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703452    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05282817-c06b-4790-bb94-cb414c255936-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703495    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.703832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703553    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.704004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703857    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-socket\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.704004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.703883    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-data\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.704189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.704146    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05282817-c06b-4790-bb94-cb414c255936-istiod-ca-cert\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.706011 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.705975    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-envoy\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.706119 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.706015    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05282817-c06b-4790-bb94-cb414c255936-istio-podinfo\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.760493 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.760455    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-istio-token\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.760670 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.760498    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpwz4\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-kube-api-access-bpwz4\") pod \"e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") " pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.766416 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.766384    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:47.805333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805291    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrvzv\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-kube-api-access-rrvzv\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.805534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805344    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.805534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805376    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.805534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805417    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.805534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805494    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.805766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805535    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.805766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805557    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.805766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805602    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.805766 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805668    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.806018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805906    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-certs\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.806018 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.805927    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-data\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.806281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.806257    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istiod-ca-cert\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.806376 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.806295    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.806558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.806530    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-credential-socket\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.808269 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.808247    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-envoy\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.808393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.808377    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-podinfo\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.863310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.863280    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-token\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.863903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.863878    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrvzv\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-kube-api-access-rrvzv\") pod \"e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") " pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:47.953242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:47.953163    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:48.137214 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.137177    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:48.198170 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.198133    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"]
Nov 02 03:45:48.199968 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:45:48.199789    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05282817_c06b_4790_bb94_cb414c255936.slice/crio-d11d4ccc65b778279786dde4021002553d355ee52d57a8017522ca3982231d4e WatchSource:0}: Error finding container d11d4ccc65b778279786dde4021002553d355ee52d57a8017522ca3982231d4e: Status 404 returned error can't find the container with id d11d4ccc65b778279786dde4021002553d355ee52d57a8017522ca3982231d4e
Nov 02 03:45:48.206760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.206727    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:48.206842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.206817    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:48.206904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.206866    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:48.527156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.527128    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"]
Nov 02 03:45:48.527893 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:45:48.527867    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231e6700_1eb8_4578_ae18_bfbd14ae0b55.slice/crio-11c1da744770e5dbf0fdb69e52bc080d5036c2fd2da3204ed6be31ba8877b46e WatchSource:0}: Error finding container 11c1da744770e5dbf0fdb69e52bc080d5036c2fd2da3204ed6be31ba8877b46e: Status 404 returned error can't find the container with id 11c1da744770e5dbf0fdb69e52bc080d5036c2fd2da3204ed6be31ba8877b46e
Nov 02 03:45:48.530486 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.530450    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:48.530591 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.530518    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:48.530591 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.530548    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:45:48.914709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:48.913400    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:49.078396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.078349    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:49.253108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.251951    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" event={"ID":"05282817-c06b-4790-bb94-cb414c255936","Type":"ContainerStarted","Data":"5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a"}
Nov 02 03:45:49.253108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.252019    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" event={"ID":"05282817-c06b-4790-bb94-cb414c255936","Type":"ContainerStarted","Data":"d11d4ccc65b778279786dde4021002553d355ee52d57a8017522ca3982231d4e"}
Nov 02 03:45:49.255178 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.255145    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" event={"ID":"231e6700-1eb8-4578-ae18-bfbd14ae0b55","Type":"ContainerStarted","Data":"f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968"}
Nov 02 03:45:49.255353 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.255195    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" event={"ID":"231e6700-1eb8-4578-ae18-bfbd14ae0b55","Type":"ContainerStarted","Data":"11c1da744770e5dbf0fdb69e52bc080d5036c2fd2da3204ed6be31ba8877b46e"}
Nov 02 03:45:49.373186 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.373136    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" podStartSLOduration=2.373123148 podStartE2EDuration="2.373123148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:45:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:45:49.369452368 +0000 UTC m=+3493.164800933" watchObservedRunningTime="2026-11-02 03:45:49.373123148 +0000 UTC m=+3493.168471713"
Nov 02 03:45:49.767175 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.767140    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:49.768602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.768577    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 03:45:49.768743 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:49.768651    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 03:45:50.138705 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:50.138362    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:50.140643 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:50.139992    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Nov 02 03:45:50.140643 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:50.140038    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Nov 02 03:45:50.766787 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:50.766754    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 03:45:50.766938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:50.766810    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 03:45:51.138121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:51.138028    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Nov 02 03:45:51.138121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:51.138095    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Nov 02 03:45:51.767726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:51.767686    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 03:45:51.767899 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:51.767746    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 03:45:52.137841 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:52.137746    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Nov 02 03:45:52.137841 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:52.137822    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Nov 02 03:45:52.501711 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:52.495617    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:45:52.767189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:52.767104    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 03:45:52.767189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:52.767171    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 03:45:53.138526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:53.138433    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Nov 02 03:45:53.138526 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:53.138491    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Nov 02 03:45:53.767072 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:53.767037    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 03:45:53.767571 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:53.767110    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 03:45:54.138160 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:54.138067    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Nov 02 03:45:54.139024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:54.138992    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Nov 02 03:45:54.767501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:54.767467    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 03:45:54.767920 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:54.767517    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 03:45:55.137846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:55.137752    4407 patch_prober.go:29] interesting pod/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Nov 02 03:45:55.137846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:55.137808    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Nov 02 03:45:55.907524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:55.907487    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:55.908115 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:55.908080    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:55.908926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:55.908904    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:45:56.079530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:56.079466    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" podStartSLOduration=9.079447501 podStartE2EDuration="9.079447501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:45:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:45:49.542151374 +0000 UTC m=+3493.337499944" watchObservedRunningTime="2026-11-02 03:45:56.079447501 +0000 UTC m=+3499.874796066"
Nov 02 03:45:56.410039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:56.410013    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:56.410720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:56.410690    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:56.411524 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:56.411506    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:45:57.282805 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:45:57.282774    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:02.060823 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:02.060790    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:05.188676 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:46:05.188638    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:46:06.135904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:06.135863    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:46:06.786095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:06.786056    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:11.575746 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:11.575715    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:13.865671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:13.865639    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:16.840387 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:16.840357    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:22.003009 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:22.002978    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:22.162056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:22.162012    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:23.690904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:23.690873    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"]
Nov 02 03:46:23.800696 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:23.800651    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"]
Nov 02 03:46:24.152667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:24.152605    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:24.306947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:24.306912    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:27.300975 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:27.300924    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:27.477155 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:27.477124    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:27.657462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:27.657367    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:28.691195 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:28.691155    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb" podUID="dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" containerName="payload-processing" containerID="cri-o://59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041" gracePeriod=25
Nov 02 03:46:28.801027 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:28.800967    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc" podUID="f182996f-880d-4999-a72d-9f72183578e5" containerName="payload-pre-processing" containerID="cri-o://dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc" gracePeriod=25
Nov 02 03:46:29.372942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:29.372904    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc" podUID="f182996f-880d-4999-a72d-9f72183578e5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 03:46:29.373193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:29.372963    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb" podUID="dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 03:46:32.704192 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:32.704156    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:32.858513 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:32.858476    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:33.026379 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:33.026342    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:35.189661 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:46:35.189605    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:46:37.980547 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:37.980518    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:39.372284 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:39.372251    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc" podUID="f182996f-880d-4999-a72d-9f72183578e5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 03:46:39.374610 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:39.372262    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb" podUID="dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 03:46:42.627259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:42.627221    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:42.782924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:42.782869    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:42.939356 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:42.939273    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:47.556775 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:47.556743    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:48.310566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:48.310529    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:48.334648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:48.324599    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:46:49.015004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.014980    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:46:49.155689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.155665    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:46:49.161091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.161070    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-plugins-config-volume\") pod \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\" (UID: \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\") "
Nov 02 03:46:49.161213 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.161167    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-kube-api-access-4qnsl\" (UniqueName: \"kubernetes.io/projected/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-kube-api-access-4qnsl\") pod \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\" (UID: \"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a\") "
Nov 02 03:46:49.161494 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.161471    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-plugins-config-volume" pod "dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" (UID: "dcb99f1f-a3fd-49d1-9e5b-468d73aac84a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:46:49.161822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.161800    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:46:49.163650 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.163612    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-kube-api-access-4qnsl" pod "dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" (UID: "dcb99f1f-a3fd-49d1-9e5b-468d73aac84a"). InnerVolumeSpecName "kube-api-access-4qnsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:46:49.263054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.263023    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f182996f-880d-4999-a72d-9f72183578e5-kube-api-access-mwz6f\" (UniqueName: \"kubernetes.io/projected/f182996f-880d-4999-a72d-9f72183578e5-kube-api-access-mwz6f\") pod \"f182996f-880d-4999-a72d-9f72183578e5\" (UID: \"f182996f-880d-4999-a72d-9f72183578e5\") "
Nov 02 03:46:49.263054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.263059    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f182996f-880d-4999-a72d-9f72183578e5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f182996f-880d-4999-a72d-9f72183578e5-plugins-config-volume\") pod \"f182996f-880d-4999-a72d-9f72183578e5\" (UID: \"f182996f-880d-4999-a72d-9f72183578e5\") "
Nov 02 03:46:49.263508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.263485    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f182996f-880d-4999-a72d-9f72183578e5-plugins-config-volume" pod "f182996f-880d-4999-a72d-9f72183578e5" (UID: "f182996f-880d-4999-a72d-9f72183578e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:46:49.263508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.263501    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qnsl\" (UniqueName: \"kubernetes.io/projected/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a-kube-api-access-4qnsl\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:46:49.265251 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.265230    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f182996f-880d-4999-a72d-9f72183578e5-kube-api-access-mwz6f" pod "f182996f-880d-4999-a72d-9f72183578e5" (UID: "f182996f-880d-4999-a72d-9f72183578e5"). InnerVolumeSpecName "kube-api-access-mwz6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:46:49.364979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.364946    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwz6f\" (UniqueName: \"kubernetes.io/projected/f182996f-880d-4999-a72d-9f72183578e5-kube-api-access-mwz6f\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:46:49.364979 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.364974    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f182996f-880d-4999-a72d-9f72183578e5-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:46:49.681949 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.681863    4407 generic.go:356] "Generic (PLEG): container finished" podID="f182996f-880d-4999-a72d-9f72183578e5" containerID="dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc" exitCode=0
Nov 02 03:46:49.681949 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.681904    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"
Nov 02 03:46:49.682156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.681954    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc" event={"ID":"f182996f-880d-4999-a72d-9f72183578e5","Type":"ContainerDied","Data":"dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc"}
Nov 02 03:46:49.682156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.682004    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc" event={"ID":"f182996f-880d-4999-a72d-9f72183578e5","Type":"ContainerDied","Data":"fd75e8b1b7cd0686472eb8f8848b94307e128c91fcff89b347ebe99a30fee6df"}
Nov 02 03:46:49.682156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.682033    4407 scope.go:122] "RemoveContainer" containerID="dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc"
Nov 02 03:46:49.683415 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.683392    4407 generic.go:356] "Generic (PLEG): container finished" podID="dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" containerID="59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041" exitCode=0
Nov 02 03:46:49.683497 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.683435    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"
Nov 02 03:46:49.683539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.683433    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb" event={"ID":"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a","Type":"ContainerDied","Data":"59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041"}
Nov 02 03:46:49.683582 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.683557    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb" event={"ID":"dcb99f1f-a3fd-49d1-9e5b-468d73aac84a","Type":"ContainerDied","Data":"021f5a5792f38a3957d230d45e1d57d06c9066b173e269c2aba3ecd50f1d482b"}
Nov 02 03:46:49.691898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.691883    4407 scope.go:122] "RemoveContainer" containerID="dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc"
Nov 02 03:46:49.692180 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:46:49.692156    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc\": container with ID starting with dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc not found: ID does not exist" containerID="dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc"
Nov 02 03:46:49.692254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.692190    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc"} err="failed to get container status \"dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc\": rpc error: code = NotFound desc = could not find container \"dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc\": container with ID starting with dbaea9a6d156837e35aedbb674874b56780e27c415a752138ae9080a0db59cbc not found: ID does not exist"
Nov 02 03:46:49.692254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.692212    4407 scope.go:122] "RemoveContainer" containerID="59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041"
Nov 02 03:46:49.700702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.700682    4407 scope.go:122] "RemoveContainer" containerID="59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041"
Nov 02 03:46:49.700957 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:46:49.700940    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041\": container with ID starting with 59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041 not found: ID does not exist" containerID="59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041"
Nov 02 03:46:49.701020 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.700967    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041"} err="failed to get container status \"59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041\": rpc error: code = NotFound desc = could not find container \"59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041\": container with ID starting with 59f8642e3499dc910184fbfacc6cab16abfc8b3d887045a3dfd9f04cb4509041 not found: ID does not exist"
Nov 02 03:46:49.848467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.848427    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"]
Nov 02 03:46:49.899193 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.899154    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-1b1008-5bdf9cb9f-pmtjc"]
Nov 02 03:46:49.967722 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.967646    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"]
Nov 02 03:46:49.970543 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:49.970517    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-1b1008-54d58d85d7-lxdpb"]
Nov 02 03:46:50.351725 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:50.351685    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" path="/var/lib/kubelet/pods/dcb99f1f-a3fd-49d1-9e5b-468d73aac84a/volumes"
Nov 02 03:46:50.352108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:50.352094    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f182996f-880d-4999-a72d-9f72183578e5" path="/var/lib/kubelet/pods/f182996f-880d-4999-a72d-9f72183578e5/volumes"
Nov 02 03:46:52.479453 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:52.479421    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:57.268493 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:57.268453    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:46:59.089757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:59.089684    4407 patch_prober.go:29] interesting pod/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Nov 02 03:46:59.089757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:59.089744    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Nov 02 03:46:59.765106 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:59.765067    4407 generic.go:356] "Generic (PLEG): container finished" podID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerID="1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc" exitCode=0
Nov 02 03:46:59.765300 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:59.765138    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" event={"ID":"6ac08935-e676-4ffc-ab77-4abc80198b6c","Type":"ContainerDied","Data":"1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc"}
Nov 02 03:46:59.765570 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:59.765556    4407 scope.go:122] "RemoveContainer" containerID="1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc"
Nov 02 03:46:59.767081 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:59.767048    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:46:59.767212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:59.767137    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:46:59.767212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:46:59.767185    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:47:00.772232 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:00.772200    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" event={"ID":"6ac08935-e676-4ffc-ab77-4abc80198b6c","Type":"ContainerStarted","Data":"200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f"}
Nov 02 03:47:01.421148 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:01.421114    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:47:01.421337 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:01.421314    4407 patch_prober.go:29] interesting pod/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Nov 02 03:47:01.421383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:01.421357    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Nov 02 03:47:02.007198 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:02.007159    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:02.421045 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:02.420957    4407 patch_prober.go:29] interesting pod/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Nov 02 03:47:02.421045 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:02.421018    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Nov 02 03:47:03.421325 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:03.421294    4407 patch_prober.go:29] interesting pod/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Nov 02 03:47:03.421822 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:03.421351    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Nov 02 03:47:04.421385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:04.421344    4407 patch_prober.go:29] interesting pod/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Nov 02 03:47:04.421834 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:04.421413    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Nov 02 03:47:05.215488 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:47:05.215391    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:47:05.421312 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:05.421279    4407 patch_prober.go:29] interesting pod/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Nov 02 03:47:05.421493 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:05.421328    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Nov 02 03:47:06.425743 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:06.425714    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:47:06.426240 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:06.426097    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:47:06.426721 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:06.426703    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 03:47:06.532911 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:06.532883    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:06.684305 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:06.684231    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:06.842172 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:06.842144    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:11.657640 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:11.657592    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:16.283697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:16.283665    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:47:16.481890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:16.481852    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:21.259649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:21.259606    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:25.998886 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:25.998842    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:30.758864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:30.758833    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:31.391055 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:31.391020    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:31.804810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:31.804763    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:32.276582 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.276543    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"]
Nov 02 03:47:32.277189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.277169    4407 cpu_manager.go:414] "containerMap: removing container" podUID="f182996f-880d-4999-a72d-9f72183578e5" containerName="payload-pre-processing"
Nov 02 03:47:32.277311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.277191    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f182996f-880d-4999-a72d-9f72183578e5" containerName="payload-pre-processing"
Nov 02 03:47:32.277311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.277261    4407 cpu_manager.go:414] "containerMap: removing container" podUID="dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" containerName="payload-processing"
Nov 02 03:47:32.277311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.277270    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" containerName="payload-processing"
Nov 02 03:47:32.277492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.277398    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcb99f1f-a3fd-49d1-9e5b-468d73aac84a" containerName="payload-processing"
Nov 02 03:47:32.277492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.277417    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="f182996f-880d-4999-a72d-9f72183578e5" containerName="payload-pre-processing"
Nov 02 03:47:32.281689 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.281668    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.357709 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.356904    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-147104-openshift-default-dockercfg-jhs8n\""
Nov 02 03:47:32.398007 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.394908    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:32.406505 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.406469    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"]
Nov 02 03:47:32.466615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.466567    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-socket\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.466887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.466672    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-certs\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.466887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.466763    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istiod-ca-cert\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.466887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.466800    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7k9m\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-kube-api-access-q7k9m\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.467089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.466898    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-envoy\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.467089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.466996    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-podinfo\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.467089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.467040    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-token\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.467089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.467083    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-data\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.467313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.467109    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-credential-socket\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.568457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568353    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-data\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.568457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568411    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-credential-socket\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.568457 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568450    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-socket\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.568862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568517    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-certs\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.568862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568613    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istiod-ca-cert\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.568862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568691    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7k9m\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-kube-api-access-q7k9m\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.568862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568773    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-envoy\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.569077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568925    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-credential-socket\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.569077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.568976    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-data\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.569077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.569017    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-socket\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.569077 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.569044    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-certs\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.569288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.569119    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-podinfo\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.569288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.569157    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-token\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.569418 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.569346    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istiod-ca-cert\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.571145 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.571120    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-envoy\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.571637 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.571606    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-podinfo\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.626855 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.626821    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-token\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.627177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.627154    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7k9m\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-kube-api-access-q7k9m\") pod \"e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") " pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:32.669855 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.669826    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:32.894905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:32.894832    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:33.320407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:33.320369    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"]
Nov 02 03:47:33.320949 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:47:33.320925    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf9995a_bd67_4765_b6d8_0e1cfeced751.slice/crio-a62bb20e58705cdd029012e267a0b2f9396424008f1ee525a0bbaa33f96f93f7 WatchSource:0}: Error finding container a62bb20e58705cdd029012e267a0b2f9396424008f1ee525a0bbaa33f96f93f7: Status 404 returned error can't find the container with id a62bb20e58705cdd029012e267a0b2f9396424008f1ee525a0bbaa33f96f93f7
Nov 02 03:47:33.323336 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:33.323300    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:47:33.323671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:33.323371    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:47:33.323671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:33.323400    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:47:33.991538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:33.991503    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" event={"ID":"ebf9995a-bd67-4765-b6d8-0e1cfeced751","Type":"ContainerStarted","Data":"26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4"}
Nov 02 03:47:33.992050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:33.991562    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" event={"ID":"ebf9995a-bd67-4765-b6d8-0e1cfeced751","Type":"ContainerStarted","Data":"a62bb20e58705cdd029012e267a0b2f9396424008f1ee525a0bbaa33f96f93f7"}
Nov 02 03:47:34.113386 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:34.113322    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podStartSLOduration=2.113301497 podStartE2EDuration="2.113301497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:47:34.109794662 +0000 UTC m=+3597.905143227" watchObservedRunningTime="2026-11-02 03:47:34.113301497 +0000 UTC m=+3597.908650063"
Nov 02 03:47:34.895012 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:34.894979    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:34.900647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:34.900598    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:34.997614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:34.997589    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:34.998471 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:34.998455    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:47:35.192527 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:47:35.192444    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:47:36.245489 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:36.245383    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:37.887642 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:37.887586    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:38.107945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:38.107913    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:38.266966 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:38.266931    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:38.431403 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:38.431362    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:38.830088 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:38.830058    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:39.413247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:39.413195    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:39.765512 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:39.765476    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:41.475031 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.474991    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:41.659668 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.659635    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:41.821409 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.821379    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:41.867164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.867130    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"]
Nov 02 03:47:41.871416 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.871395    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:41.907313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.907284    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cff9b9b-3a83-4704-a39a-c39f505d8686-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd\" (UID: \"1cff9b9b-3a83-4704-a39a-c39f505d8686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:41.907463 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.907367    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2rmr\" (UniqueName: \"kubernetes.io/projected/1cff9b9b-3a83-4704-a39a-c39f505d8686-kube-api-access-p2rmr\") pod \"payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd\" (UID: \"1cff9b9b-3a83-4704-a39a-c39f505d8686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:41.925730 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.925689    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"]
Nov 02 03:47:41.981654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.974957    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-147104\""
Nov 02 03:47:41.981654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:41.975104    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-147104-dockercfg-sxh86\""
Nov 02 03:47:42.028503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.027811    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cff9b9b-3a83-4704-a39a-c39f505d8686-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd\" (UID: \"1cff9b9b-3a83-4704-a39a-c39f505d8686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:42.028503 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.027936    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2rmr\" (UniqueName: \"kubernetes.io/projected/1cff9b9b-3a83-4704-a39a-c39f505d8686-kube-api-access-p2rmr\") pod \"payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd\" (UID: \"1cff9b9b-3a83-4704-a39a-c39f505d8686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:42.029050 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.029017    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cff9b9b-3a83-4704-a39a-c39f505d8686-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd\" (UID: \"1cff9b9b-3a83-4704-a39a-c39f505d8686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:42.034963 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.034930    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"]
Nov 02 03:47:42.039960 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.039898    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:42.086917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.086838    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2rmr\" (UniqueName: \"kubernetes.io/projected/1cff9b9b-3a83-4704-a39a-c39f505d8686-kube-api-access-p2rmr\") pod \"payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd\" (UID: \"1cff9b9b-3a83-4704-a39a-c39f505d8686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:42.087396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.087371    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"]
Nov 02 03:47:42.129134 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.129092    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad206f87-2062-48a7-b2c6-df915c4a7a63-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6\" (UID: \"ad206f87-2062-48a7-b2c6-df915c4a7a63\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:42.129478 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.129445    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv4mz\" (UniqueName: \"kubernetes.io/projected/ad206f87-2062-48a7-b2c6-df915c4a7a63-kube-api-access-jv4mz\") pod \"payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6\" (UID: \"ad206f87-2062-48a7-b2c6-df915c4a7a63\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:42.185757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.185724    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:42.230919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.230881    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad206f87-2062-48a7-b2c6-df915c4a7a63-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6\" (UID: \"ad206f87-2062-48a7-b2c6-df915c4a7a63\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:42.231089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.231072    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv4mz\" (UniqueName: \"kubernetes.io/projected/ad206f87-2062-48a7-b2c6-df915c4a7a63-kube-api-access-jv4mz\") pod \"payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6\" (UID: \"ad206f87-2062-48a7-b2c6-df915c4a7a63\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:42.231786 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.231763    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad206f87-2062-48a7-b2c6-df915c4a7a63-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6\" (UID: \"ad206f87-2062-48a7-b2c6-df915c4a7a63\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:42.288849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.288813    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv4mz\" (UniqueName: \"kubernetes.io/projected/ad206f87-2062-48a7-b2c6-df915c4a7a63-kube-api-access-jv4mz\") pod \"payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6\" (UID: \"ad206f87-2062-48a7-b2c6-df915c4a7a63\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:42.354317 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.354237    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:42.564154 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.564124    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"]
Nov 02 03:47:42.564860 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:47:42.564833    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cff9b9b_3a83_4704_a39a_c39f505d8686.slice/crio-15d8e34d94f58a6191458c5a1e942fe371add7866368d31efb68900cf2998e42 WatchSource:0}: Error finding container 15d8e34d94f58a6191458c5a1e942fe371add7866368d31efb68900cf2998e42: Status 404 returned error can't find the container with id 15d8e34d94f58a6191458c5a1e942fe371add7866368d31efb68900cf2998e42
Nov 02 03:47:42.727425 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:42.727396    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"]
Nov 02 03:47:42.728153 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:47:42.728121    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad206f87_2062_48a7_b2c6_df915c4a7a63.slice/crio-86a92a10ff9c59cd9e5d1a8c4dba60293fde6dfdebc164fcbd5f6c6036a25f5c WatchSource:0}: Error finding container 86a92a10ff9c59cd9e5d1a8c4dba60293fde6dfdebc164fcbd5f6c6036a25f5c: Status 404 returned error can't find the container with id 86a92a10ff9c59cd9e5d1a8c4dba60293fde6dfdebc164fcbd5f6c6036a25f5c
Nov 02 03:47:43.049044 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.049006    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" event={"ID":"1cff9b9b-3a83-4704-a39a-c39f505d8686","Type":"ContainerStarted","Data":"7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb"}
Nov 02 03:47:43.049243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.049067    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:43.049243 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.049090    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" event={"ID":"1cff9b9b-3a83-4704-a39a-c39f505d8686","Type":"ContainerStarted","Data":"15d8e34d94f58a6191458c5a1e942fe371add7866368d31efb68900cf2998e42"}
Nov 02 03:47:43.050600 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.050577    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" event={"ID":"ad206f87-2062-48a7-b2c6-df915c4a7a63","Type":"ContainerStarted","Data":"4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481"}
Nov 02 03:47:43.050733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.050640    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" event={"ID":"ad206f87-2062-48a7-b2c6-df915c4a7a63","Type":"ContainerStarted","Data":"86a92a10ff9c59cd9e5d1a8c4dba60293fde6dfdebc164fcbd5f6c6036a25f5c"}
Nov 02 03:47:43.050733 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.050677    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:43.165531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.165464    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" podStartSLOduration=2.16545212 podStartE2EDuration="2.16545212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:47:43.162193192 +0000 UTC m=+3606.957541758" watchObservedRunningTime="2026-11-02 03:47:43.16545212 +0000 UTC m=+3606.960800684"
Nov 02 03:47:43.279977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.279917    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" podStartSLOduration=2.279898705 podStartE2EDuration="2.279898705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:47:43.276604604 +0000 UTC m=+3607.071953170" watchObservedRunningTime="2026-11-02 03:47:43.279898705 +0000 UTC m=+3607.075247271"
Nov 02 03:47:43.890102 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:43.890071    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:44.089846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:44.089813    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:44.248084 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:44.248056    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:44.673868 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:44.673786    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:45.098281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.098245    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:45.569835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.569797    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"]
Nov 02 03:47:45.573886 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.573865    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.654646 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.645720    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-3b27a5-openshift-default-dockercfg-6krvj\""
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.684588    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-token\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.684685    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-data\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.684724    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-envoy\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.684764    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t62jl\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-kube-api-access-t62jl\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.684827    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-socket\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.684894    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-credential-socket\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.684990    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-certs\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.685015    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istiod-ca-cert\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.686669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.685127    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-podinfo\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.711514 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.704864    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"]
Nov 02 03:47:45.716443 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.714668    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:45.786352 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786313    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-socket\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.786538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786414    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-credential-socket\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.786538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786523    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-certs\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.786758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786553    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istiod-ca-cert\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.786758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786679    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-podinfo\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.786758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786732    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-token\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.787002 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786777    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-socket\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.787002 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786850    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-credential-socket\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.787002 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786900    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-data\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.787002 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.786905    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-certs\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.787234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.787010    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-envoy\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.787234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.787056    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t62jl\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-kube-api-access-t62jl\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.787234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.787193    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-data\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.787406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.787368    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istiod-ca-cert\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.789893 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.789866    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-envoy\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.790021 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.790001    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-podinfo\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.845054 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.844982    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-token\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.845200 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.845176    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t62jl\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-kube-api-access-t62jl\") pod \"e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") " pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:45.887896 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.887850    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:45.888072 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:45.888001    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:46.051760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:46.051724    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:46.325344 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:46.325316    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"]
Nov 02 03:47:46.326183 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:47:46.326156    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0fc959_11b3_40cc_ab7c_6dd2474b076d.slice/crio-f77a9d078a4e2b2f02450c45ec38abb03162b9d168a34152ea1b72ffb22b432b WatchSource:0}: Error finding container f77a9d078a4e2b2f02450c45ec38abb03162b9d168a34152ea1b72ffb22b432b: Status 404 returned error can't find the container with id f77a9d078a4e2b2f02450c45ec38abb03162b9d168a34152ea1b72ffb22b432b
Nov 02 03:47:46.328869 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:46.328835    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:47:46.328947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:46.328900    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:47:46.328947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:46.328931    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:47:47.078470 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:47.078430    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" event={"ID":"df0fc959-11b3-40cc-ab7c-6dd2474b076d","Type":"ContainerStarted","Data":"5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a"}
Nov 02 03:47:47.078686 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:47.078495    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" event={"ID":"df0fc959-11b3-40cc-ab7c-6dd2474b076d","Type":"ContainerStarted","Data":"f77a9d078a4e2b2f02450c45ec38abb03162b9d168a34152ea1b72ffb22b432b"}
Nov 02 03:47:47.297515 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:47.297452    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podStartSLOduration=2.297437794 podStartE2EDuration="2.297437794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:47:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:47:47.246479049 +0000 UTC m=+3611.041827614" watchObservedRunningTime="2026-11-02 03:47:47.297437794 +0000 UTC m=+3611.092786359"
Nov 02 03:47:47.589697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:47.589655    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:47.888607 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:47.888528    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:47.894432 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:47.894407    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:48.085231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:48.085205    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:48.086205 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:48.086185    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:47:48.519699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:48.519667    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:48.671915 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:48.671877    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:48.823716 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:48.823554    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:50.661803 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:50.661775    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:50.882825 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:50.882785    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:51.045879 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:51.045848    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:53.042804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:53.042756    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:54.061462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:54.061435    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 03:47:54.061904 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:54.061504    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 03:47:55.157789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:55.157755    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:55.591396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:55.591339    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:55.878660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:55.878547    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:56.048882 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.048846    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:56.254484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.254454    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"]
Nov 02 03:47:56.258827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.258805    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:56.315637 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.315604    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"]
Nov 02 03:47:56.364511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.362893    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-3b27a5-dockercfg-xjwsk\""
Nov 02 03:47:56.364511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.362956    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-3b27a5\""
Nov 02 03:47:56.432491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.430262    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"]
Nov 02 03:47:56.438643 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.434491    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfdd5fd-392d-49fc-b7e3-98d12700017e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl\" (UID: \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:56.438643 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.434829    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfqgf\" (UniqueName: \"kubernetes.io/projected/bcfdd5fd-392d-49fc-b7e3-98d12700017e-kube-api-access-xfqgf\") pod \"payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl\" (UID: \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:56.438643 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.435888    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:56.490544 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.490517    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"]
Nov 02 03:47:56.535804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.535722    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgwlz\" (UniqueName: \"kubernetes.io/projected/32472e85-b15a-469c-9a32-efb1b361c80d-kube-api-access-dgwlz\") pod \"payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp\" (UID: \"32472e85-b15a-469c-9a32-efb1b361c80d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:56.535804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.535778    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfqgf\" (UniqueName: \"kubernetes.io/projected/bcfdd5fd-392d-49fc-b7e3-98d12700017e-kube-api-access-xfqgf\") pod \"payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl\" (UID: \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:56.535996 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.535939    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32472e85-b15a-469c-9a32-efb1b361c80d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp\" (UID: \"32472e85-b15a-469c-9a32-efb1b361c80d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:56.536065 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.536051    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfdd5fd-392d-49fc-b7e3-98d12700017e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl\" (UID: \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:56.536817 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.536800    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfdd5fd-392d-49fc-b7e3-98d12700017e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl\" (UID: \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:56.593443 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.593407    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfqgf\" (UniqueName: \"kubernetes.io/projected/bcfdd5fd-392d-49fc-b7e3-98d12700017e-kube-api-access-xfqgf\") pod \"payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl\" (UID: \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:56.637769 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.637732    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgwlz\" (UniqueName: \"kubernetes.io/projected/32472e85-b15a-469c-9a32-efb1b361c80d-kube-api-access-dgwlz\") pod \"payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp\" (UID: \"32472e85-b15a-469c-9a32-efb1b361c80d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:56.637918 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.637818    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32472e85-b15a-469c-9a32-efb1b361c80d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp\" (UID: \"32472e85-b15a-469c-9a32-efb1b361c80d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:56.638492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.638471    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32472e85-b15a-469c-9a32-efb1b361c80d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp\" (UID: \"32472e85-b15a-469c-9a32-efb1b361c80d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:56.697575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.697547    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgwlz\" (UniqueName: \"kubernetes.io/projected/32472e85-b15a-469c-9a32-efb1b361c80d-kube-api-access-dgwlz\") pod \"payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp\" (UID: \"32472e85-b15a-469c-9a32-efb1b361c80d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:56.758459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.758423    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:56.840056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.839950    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:56.885238 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:56.884968    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:57.139562 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:57.139469    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:47:57.147790 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:47:57.147761    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32472e85_b15a_469c_9a32_efb1b361c80d.slice/crio-260e05b5176d201075026055e504cea3f28da969cf46a5c2c03927b8fc1b0eef WatchSource:0}: Error finding container 260e05b5176d201075026055e504cea3f28da969cf46a5c2c03927b8fc1b0eef: Status 404 returned error can't find the container with id 260e05b5176d201075026055e504cea3f28da969cf46a5c2c03927b8fc1b0eef
Nov 02 03:47:57.149130 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:57.149106    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"]
Nov 02 03:47:57.278161 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:57.278135    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"]
Nov 02 03:47:57.278830 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:47:57.278804    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfdd5fd_392d_49fc_b7e3_98d12700017e.slice/crio-0c01afcf648315b1e5348df1a45a785b1994700ff303f032b126a2e5b6775ab2 WatchSource:0}: Error finding container 0c01afcf648315b1e5348df1a45a785b1994700ff303f032b126a2e5b6775ab2: Status 404 returned error can't find the container with id 0c01afcf648315b1e5348df1a45a785b1994700ff303f032b126a2e5b6775ab2
Nov 02 03:47:58.156300 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.156267    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp" event={"ID":"32472e85-b15a-469c-9a32-efb1b361c80d","Type":"ContainerStarted","Data":"750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00"}
Nov 02 03:47:58.156508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.156351    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp" event={"ID":"32472e85-b15a-469c-9a32-efb1b361c80d","Type":"ContainerStarted","Data":"260e05b5176d201075026055e504cea3f28da969cf46a5c2c03927b8fc1b0eef"}
Nov 02 03:47:58.156508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.156377    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:47:58.158053 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.158020    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl" event={"ID":"bcfdd5fd-392d-49fc-b7e3-98d12700017e","Type":"ContainerStarted","Data":"81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00"}
Nov 02 03:47:58.158216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.158076    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl" event={"ID":"bcfdd5fd-392d-49fc-b7e3-98d12700017e","Type":"ContainerStarted","Data":"0c01afcf648315b1e5348df1a45a785b1994700ff303f032b126a2e5b6775ab2"}
Nov 02 03:47:58.158216 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.158107    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:47:58.273657 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.273595    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp" podStartSLOduration=2.273582963 podStartE2EDuration="2.273582963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:47:58.271010828 +0000 UTC m=+3622.066359393" watchObservedRunningTime="2026-11-02 03:47:58.273582963 +0000 UTC m=+3622.068931544"
Nov 02 03:47:58.387100 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.387041    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl" podStartSLOduration=2.387025337 podStartE2EDuration="2.387025337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:47:58.386285588 +0000 UTC m=+3622.181634154" watchObservedRunningTime="2026-11-02 03:47:58.387025337 +0000 UTC m=+3622.182373902"
Nov 02 03:47:58.535351 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:47:58.535323    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:01.428898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:01.428872    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:01.578859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:01.578823    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:48:02.914101 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:02.914070    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:03.695267 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:03.695237    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:04.669078 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:04.669049    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:04.855877 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:04.855845    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:05.060095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.060062    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"]
Nov 02 03:48:05.065566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.065543    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:05.121749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.121719    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"]
Nov 02 03:48:05.143174 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.143141    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed0c132-c992-4390-923a-5cbeadffa424-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd\" (UID: \"eed0c132-c992-4390-923a-5cbeadffa424\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:05.143345 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.143286    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn4gh\" (UniqueName: \"kubernetes.io/projected/eed0c132-c992-4390-923a-5cbeadffa424-kube-api-access-qn4gh\") pod \"payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd\" (UID: \"eed0c132-c992-4390-923a-5cbeadffa424\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:05.168805 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.168774    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-3d14ee\""
Nov 02 03:48:05.169159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.169138    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-3d14ee-dockercfg-75r8p\""
Nov 02 03:48:05.235878 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:48:05.235838    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:48:05.250745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.245202    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed0c132-c992-4390-923a-5cbeadffa424-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd\" (UID: \"eed0c132-c992-4390-923a-5cbeadffa424\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:05.250745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.245318    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn4gh\" (UniqueName: \"kubernetes.io/projected/eed0c132-c992-4390-923a-5cbeadffa424-kube-api-access-qn4gh\") pod \"payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd\" (UID: \"eed0c132-c992-4390-923a-5cbeadffa424\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:05.250745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.247442    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed0c132-c992-4390-923a-5cbeadffa424-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd\" (UID: \"eed0c132-c992-4390-923a-5cbeadffa424\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:05.304306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.304271    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn4gh\" (UniqueName: \"kubernetes.io/projected/eed0c132-c992-4390-923a-5cbeadffa424-kube-api-access-qn4gh\") pod \"payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd\" (UID: \"eed0c132-c992-4390-923a-5cbeadffa424\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:05.380699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.380600    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:05.754744 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:48:05.754710    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed0c132_c992_4390_923a_5cbeadffa424.slice/crio-f32b86f799b17023a1e4fef1cd8e96f05aa8d57088724a8c8ec0928d34c7c30d WatchSource:0}: Error finding container f32b86f799b17023a1e4fef1cd8e96f05aa8d57088724a8c8ec0928d34c7c30d: Status 404 returned error can't find the container with id f32b86f799b17023a1e4fef1cd8e96f05aa8d57088724a8c8ec0928d34c7c30d
Nov 02 03:48:05.755412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:05.755382    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"]
Nov 02 03:48:06.290231 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:06.290197    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd" event={"ID":"eed0c132-c992-4390-923a-5cbeadffa424","Type":"ContainerStarted","Data":"4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409"}
Nov 02 03:48:06.290419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:06.290257    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd" event={"ID":"eed0c132-c992-4390-923a-5cbeadffa424","Type":"ContainerStarted","Data":"f32b86f799b17023a1e4fef1cd8e96f05aa8d57088724a8c8ec0928d34c7c30d"}
Nov 02 03:48:06.290419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:06.290294    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:06.405930 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:06.405870    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd" podStartSLOduration=1.405850426 podStartE2EDuration="1.405850426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:48:06.404738134 +0000 UTC m=+3630.200086700" watchObservedRunningTime="2026-11-02 03:48:06.405850426 +0000 UTC m=+3630.201198992"
Nov 02 03:48:08.967710 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:08.967677    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:09.167808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:09.167783    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 03:48:09.167998 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:09.167850    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 03:48:11.985145 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:11.985118    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:12.136698 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:12.136645    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:13.857939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:13.857904    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:17.298859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:17.298831    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 03:48:18.654128 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:18.654053    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:22.846972 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:22.846939    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:24.330584 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:24.330545    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:24.576467 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:24.576430    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:24.738271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:24.738194    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:25.209906 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:25.209874    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:29.776781 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:29.776744    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:30.249753 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:30.249713    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:35.022830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:35.022793    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:35.251647 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:48:35.250878    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:48:35.416932 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:35.416841    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:48:39.965349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:39.965295    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:44.609414 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:44.609382    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:44.760134 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:44.760084    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:44.913851 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:44.913752    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:48.984538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:48.984495    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:49.601158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:49.601096    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:54.222767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:54.222733    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:54.391651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:54.391607    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:54.547368 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:54.547319    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:48:59.171322 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:48:59.171287    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:00.311618 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:00.311585    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:03.968652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:03.966952    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:05.212787 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:49:05.212751    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:49:07.109566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:07.109533    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:07.293764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:07.293713    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:08.606981 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:08.606948    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:08.762912 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:08.762882    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:08.965145 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:08.965046    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:13.037111 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.037076    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:13.193184 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.193152    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:13.360335 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.360265    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:13.603615 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.603582    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:13.765898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.765871    4407 generic.go:356] "Generic (PLEG): container finished" podID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerID="26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4" exitCode=0
Nov 02 03:49:13.766064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.765947    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" event={"ID":"ebf9995a-bd67-4765-b6d8-0e1cfeced751","Type":"ContainerDied","Data":"26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4"}
Nov 02 03:49:13.766393 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.766374    4407 scope.go:122] "RemoveContainer" containerID="26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4"
Nov 02 03:49:13.768091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.768060    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:49:13.768161 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.768132    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:49:13.768202 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:13.768171    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:49:14.773885 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:14.773846    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" event={"ID":"ebf9995a-bd67-4765-b6d8-0e1cfeced751","Type":"ContainerStarted","Data":"19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72"}
Nov 02 03:49:14.902086 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:14.902052    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:49:14.902504 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:14.902409    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Nov 02 03:49:14.902706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:14.902684    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Nov 02 03:49:15.903649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:15.903089    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Nov 02 03:49:15.903649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:15.903164    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Nov 02 03:49:16.900049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:16.900005    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Nov 02 03:49:16.900395 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:16.900365    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Nov 02 03:49:17.905015 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:17.904976    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Nov 02 03:49:17.905666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:17.905616    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Nov 02 03:49:18.804379 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:18.804336    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:18.906067 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:18.906022    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Nov 02 03:49:18.906570 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:18.906084    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Nov 02 03:49:19.895563 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:19.895522    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Nov 02 03:49:19.895782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:19.895591    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Nov 02 03:49:19.997856 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:19.997828    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:49:20.900785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:20.900756    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:49:20.901843 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:20.901823    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 03:49:23.619988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:23.619955    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:23.725283 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:23.725251    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:49:23.858947 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:23.858908    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:24.003303 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:24.003276    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:25.558186 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:25.558154    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"]
Nov 02 03:49:25.825423 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:25.825318    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:25.969883 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:25.969851    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:28.722761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:28.722723    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:30.559158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.559087    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf" podUID="7abfea6e-c9e2-4c60-9793-be526dcedfd4" containerName="payload-pre-processing" containerID="cri-o://373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa" gracePeriod=25
Nov 02 03:49:30.858264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.858232    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:49:30.952269 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.952235    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7abfea6e-c9e2-4c60-9793-be526dcedfd4-kube-api-access-74zqd\" (UniqueName: \"kubernetes.io/projected/7abfea6e-c9e2-4c60-9793-be526dcedfd4-kube-api-access-74zqd\") pod \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\" (UID: \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\") "
Nov 02 03:49:30.952459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.952387    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7abfea6e-c9e2-4c60-9793-be526dcedfd4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7abfea6e-c9e2-4c60-9793-be526dcedfd4-plugins-config-volume\") pod \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\" (UID: \"7abfea6e-c9e2-4c60-9793-be526dcedfd4\") "
Nov 02 03:49:30.952777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.952751    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7abfea6e-c9e2-4c60-9793-be526dcedfd4-plugins-config-volume" pod "7abfea6e-c9e2-4c60-9793-be526dcedfd4" (UID: "7abfea6e-c9e2-4c60-9793-be526dcedfd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:49:30.952919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.952902    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7abfea6e-c9e2-4c60-9793-be526dcedfd4-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:49:30.954660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.954643    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7abfea6e-c9e2-4c60-9793-be526dcedfd4-kube-api-access-74zqd" pod "7abfea6e-c9e2-4c60-9793-be526dcedfd4" (UID: "7abfea6e-c9e2-4c60-9793-be526dcedfd4"). InnerVolumeSpecName "kube-api-access-74zqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:49:30.970945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.970915    4407 generic.go:356] "Generic (PLEG): container finished" podID="7abfea6e-c9e2-4c60-9793-be526dcedfd4" containerID="373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa" exitCode=0
Nov 02 03:49:30.971071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.970958    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"
Nov 02 03:49:30.971071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.970991    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf" event={"ID":"7abfea6e-c9e2-4c60-9793-be526dcedfd4","Type":"ContainerDied","Data":"373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa"}
Nov 02 03:49:30.971071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.971049    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf" event={"ID":"7abfea6e-c9e2-4c60-9793-be526dcedfd4","Type":"ContainerDied","Data":"e13848fafe2ebf917bc3922e5b31d146d85742481418e680853c22a481c0647e"}
Nov 02 03:49:30.971203 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:30.971077    4407 scope.go:122] "RemoveContainer" containerID="373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa"
Nov 02 03:49:31.014214 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:31.014189    4407 scope.go:122] "RemoveContainer" containerID="373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa"
Nov 02 03:49:31.014685 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:49:31.014602    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa\": container with ID starting with 373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa not found: ID does not exist" containerID="373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa"
Nov 02 03:49:31.014826 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:31.014698    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa"} err="failed to get container status \"373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa\": rpc error: code = NotFound desc = could not find container \"373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa\": container with ID starting with 373cfb05679a14d53f332d6100175a0d852b9e2793facdae2a7425263ae9a5aa not found: ID does not exist"
Nov 02 03:49:31.054473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:31.054445    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74zqd\" (UniqueName: \"kubernetes.io/projected/7abfea6e-c9e2-4c60-9793-be526dcedfd4-kube-api-access-74zqd\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:49:31.169786 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:31.169700    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"]
Nov 02 03:49:31.172691 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:31.172667    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f68c7e-5dcfbcd4c9-lkwsf"]
Nov 02 03:49:32.353395 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:32.353359    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7abfea6e-c9e2-4c60-9793-be526dcedfd4" path="/var/lib/kubelet/pods/7abfea6e-c9e2-4c60-9793-be526dcedfd4/volumes"
Nov 02 03:49:33.540294 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:33.540260    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:34.126836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:34.126797    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 03:49:34.127024 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:34.126864    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 03:49:35.008341 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:35.008311    4407 generic.go:356] "Generic (PLEG): container finished" podID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerID="5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a" exitCode=0
Nov 02 03:49:35.008839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:35.008384    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" event={"ID":"df0fc959-11b3-40cc-ab7c-6dd2474b076d","Type":"ContainerDied","Data":"5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a"}
Nov 02 03:49:35.008839 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:35.008785    4407 scope.go:122] "RemoveContainer" containerID="5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a"
Nov 02 03:49:35.010258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:35.010228    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:49:35.010344 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:35.010312    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:49:35.010385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:35.010349    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:49:35.193746 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:49:35.193707    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:49:36.017373 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:36.017337    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" event={"ID":"df0fc959-11b3-40cc-ab7c-6dd2474b076d","Type":"ContainerStarted","Data":"68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3"}
Nov 02 03:49:36.896647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:36.896013    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:49:36.896647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:36.896183    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Nov 02 03:49:36.896647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:36.896227    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Nov 02 03:49:37.905797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:37.905755    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Nov 02 03:49:37.906335 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:37.905868    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Nov 02 03:49:38.345953 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:38.345899    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:38.894659 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:38.894030    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Nov 02 03:49:38.894659 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:38.894128    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Nov 02 03:49:39.902651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:39.897034    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Nov 02 03:49:39.902651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:39.897094    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Nov 02 03:49:40.896535 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:40.896092    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Nov 02 03:49:40.896535 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:40.896154    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Nov 02 03:49:41.893030 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:41.892997    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:49:41.893475 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:41.893451    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:49:41.894160 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:41.894142    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 03:49:43.107324 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:43.107295    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:47.267244 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:47.267210    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:48.787572 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:48.787501    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:49.025058 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:49.025020    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:49.174332 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:49.174253    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:49.335239 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:49.335207    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:49.679273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:49.679244    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:53.670716 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:53.670683    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:49:54.263734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:54.263705    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:55.548796 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:55.548762    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:55.709683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:55.709612    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:55.842274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:55.842189    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"]
Nov 02 03:49:55.842499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:55.842475    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy" containerID="cri-o://cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12" gracePeriod=30
Nov 02 03:49:55.873380 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:55.873351    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:56.941884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:56.941851    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:57.537827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:57.537798    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:57.753180 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:57.753136    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:49:59.214656 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:49:59.214587    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:00.691158 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:00.691122    4407 patch_prober.go:29] interesting pod/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:50:00.691577 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:00.691201    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:50:01.108013 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.107984    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:50:01.178560 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178525    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-podinfo\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.178783 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178598    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-certs\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.178783 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178646    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-socket\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.178783 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178688    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-envoy\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.178970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178812    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-credential-socket\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.178970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178855    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-token\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.178970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178880    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-data\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.178970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178944    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-kube-api-access-cj6p9\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-kube-api-access-cj6p9\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.178970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178955    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-certs" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:50:01.179219 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.178987    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-socket" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:50:01.179219 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179003    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0a9bc24-1360-42ed-992c-a28a22efc040-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9bc24-1360-42ed-992c-a28a22efc040-istiod-ca-cert\") pod \"f0a9bc24-1360-42ed-992c-a28a22efc040\" (UID: \"f0a9bc24-1360-42ed-992c-a28a22efc040\") "
Nov 02 03:50:01.179219 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179109    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-credential-socket" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:50:01.179506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179318    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-data" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:50:01.179506 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179503    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a9bc24-1360-42ed-992c-a28a22efc040-istiod-ca-cert" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:50:01.179699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179596    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.179699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179612    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.179699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179639    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9bc24-1360-42ed-992c-a28a22efc040-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.179699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179651    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.179699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.179665    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.181005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.180982    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-podinfo" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:50:01.181108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.181080    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-envoy" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:50:01.181761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.181736    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-token" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:50:01.181852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.181765    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-kube-api-access-cj6p9" pod "f0a9bc24-1360-42ed-992c-a28a22efc040" (UID: "f0a9bc24-1360-42ed-992c-a28a22efc040"). InnerVolumeSpecName "kube-api-access-cj6p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:50:01.211872 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.211791    4407 generic.go:356] "Generic (PLEG): container finished" podID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerID="cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12" exitCode=0
Nov 02 03:50:01.211872 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.211839    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"
Nov 02 03:50:01.212051 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.211877    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" event={"ID":"f0a9bc24-1360-42ed-992c-a28a22efc040","Type":"ContainerDied","Data":"cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12"}
Nov 02 03:50:01.212051 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.211966    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs" event={"ID":"f0a9bc24-1360-42ed-992c-a28a22efc040","Type":"ContainerDied","Data":"0844cd7080a51e3fc20b319bf2194e0e7da6e11ab78e35e1fe41ab8c6e438ff4"}
Nov 02 03:50:01.212051 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.211995    4407 scope.go:122] "RemoveContainer" containerID="cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12"
Nov 02 03:50:01.224068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.224047    4407 scope.go:122] "RemoveContainer" containerID="cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12"
Nov 02 03:50:01.224405 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:50:01.224387    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12\": container with ID starting with cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12 not found: ID does not exist" containerID="cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12"
Nov 02 03:50:01.224446 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.224415    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12"} err="failed to get container status \"cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12\": rpc error: code = NotFound desc = could not find container \"cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12\": container with ID starting with cbfc304fda6a51960886b538f204ad45b2d85e382d1807e6ae79030e51c0ef12 not found: ID does not exist"
Nov 02 03:50:01.281118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.281086    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.281118 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.281120    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.281354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.281136    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.281354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.281152    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj6p9\" (UniqueName: \"kubernetes.io/projected/f0a9bc24-1360-42ed-992c-a28a22efc040-kube-api-access-cj6p9\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:50:01.391470 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.391439    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"]
Nov 02 03:50:01.396549 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:01.396523    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-f68c7e-openshift-default-db7d68949-v6prs"]
Nov 02 03:50:02.352258 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:02.352223    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" path="/var/lib/kubelet/pods/f0a9bc24-1360-42ed-992c-a28a22efc040/volumes"
Nov 02 03:50:02.546132 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:02.546101    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:02.712793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:02.712707    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:02.949225 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:02.949198    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:04.331397 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:04.331318    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:05.188836 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:50:05.188804    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:50:05.785707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:05.785662    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:05.968807 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:05.968769    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:06.174059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.173976    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"]
Nov 02 03:50:06.174587 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.174573    4407 cpu_manager.go:414] "containerMap: removing container" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy"
Nov 02 03:50:06.174648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.174589    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy"
Nov 02 03:50:06.174688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.174649    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7abfea6e-c9e2-4c60-9793-be526dcedfd4" containerName="payload-pre-processing"
Nov 02 03:50:06.174688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.174655    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7abfea6e-c9e2-4c60-9793-be526dcedfd4" containerName="payload-pre-processing"
Nov 02 03:50:06.174760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.174731    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7abfea6e-c9e2-4c60-9793-be526dcedfd4" containerName="payload-pre-processing"
Nov 02 03:50:06.174760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.174740    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0a9bc24-1360-42ed-992c-a28a22efc040" containerName="istio-proxy"
Nov 02 03:50:06.179648 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.179603    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:06.233901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.233876    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-86f4c4\""
Nov 02 03:50:06.245549 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.245518    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czjkd\" (UniqueName: \"kubernetes.io/projected/4772f6cc-590c-4777-b7da-1b3310316f79-kube-api-access-czjkd\") pod \"payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w\" (UID: \"4772f6cc-590c-4777-b7da-1b3310316f79\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:06.245775 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.245727    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4772f6cc-590c-4777-b7da-1b3310316f79-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w\" (UID: \"4772f6cc-590c-4777-b7da-1b3310316f79\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:06.291530 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.291493    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-86f4c4-dockercfg-487lg\""
Nov 02 03:50:06.306716 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.306681    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"]
Nov 02 03:50:06.342776 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.342747    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"]
Nov 02 03:50:06.347385 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.347346    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4772f6cc-590c-4777-b7da-1b3310316f79-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w\" (UID: \"4772f6cc-590c-4777-b7da-1b3310316f79\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:06.347531 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.347466    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjkd\" (UniqueName: \"kubernetes.io/projected/4772f6cc-590c-4777-b7da-1b3310316f79-kube-api-access-czjkd\") pod \"payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w\" (UID: \"4772f6cc-590c-4777-b7da-1b3310316f79\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:06.348748 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.348707    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4772f6cc-590c-4777-b7da-1b3310316f79-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w\" (UID: \"4772f6cc-590c-4777-b7da-1b3310316f79\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:06.351342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.351322    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:06.395445 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.395411    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"]
Nov 02 03:50:06.405808 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.405782    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czjkd\" (UniqueName: \"kubernetes.io/projected/4772f6cc-590c-4777-b7da-1b3310316f79-kube-api-access-czjkd\") pod \"payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w\" (UID: \"4772f6cc-590c-4777-b7da-1b3310316f79\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:06.448491 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.448414    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrg9r\" (UniqueName: \"kubernetes.io/projected/b225fb39-617c-4075-8b02-611f6528a3b3-kube-api-access-rrg9r\") pod \"payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9\" (UID: \"b225fb39-617c-4075-8b02-611f6528a3b3\") " pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:06.448668 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.448551    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b225fb39-617c-4075-8b02-611f6528a3b3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9\" (UID: \"b225fb39-617c-4075-8b02-611f6528a3b3\") " pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:06.493354 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.493327    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:06.550249 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.550211    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrg9r\" (UniqueName: \"kubernetes.io/projected/b225fb39-617c-4075-8b02-611f6528a3b3-kube-api-access-rrg9r\") pod \"payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9\" (UID: \"b225fb39-617c-4075-8b02-611f6528a3b3\") " pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:06.550390 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.550361    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b225fb39-617c-4075-8b02-611f6528a3b3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9\" (UID: \"b225fb39-617c-4075-8b02-611f6528a3b3\") " pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:06.551484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.551313    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b225fb39-617c-4075-8b02-611f6528a3b3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9\" (UID: \"b225fb39-617c-4075-8b02-611f6528a3b3\") " pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:06.609217 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.609192    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrg9r\" (UniqueName: \"kubernetes.io/projected/b225fb39-617c-4075-8b02-611f6528a3b3-kube-api-access-rrg9r\") pod \"payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9\" (UID: \"b225fb39-617c-4075-8b02-611f6528a3b3\") " pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:06.668199 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.668165    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:06.867060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:06.867030    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"]
Nov 02 03:50:06.867385 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:50:06.867039    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4772f6cc_590c_4777_b7da_1b3310316f79.slice/crio-c0219e988bef25049ca9903715fcf87433bf5d1cc290d476312d5c0ffcde7348 WatchSource:0}: Error finding container c0219e988bef25049ca9903715fcf87433bf5d1cc290d476312d5c0ffcde7348: Status 404 returned error can't find the container with id c0219e988bef25049ca9903715fcf87433bf5d1cc290d476312d5c0ffcde7348
Nov 02 03:50:07.043413 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.043384    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"]
Nov 02 03:50:07.043528 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:50:07.043410    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb225fb39_617c_4075_8b02_611f6528a3b3.slice/crio-1a31de01c626cb02c1c2d98870bfb8e105948403725dacd8c7aa5d53f56f6d69 WatchSource:0}: Error finding container 1a31de01c626cb02c1c2d98870bfb8e105948403725dacd8c7aa5d53f56f6d69: Status 404 returned error can't find the container with id 1a31de01c626cb02c1c2d98870bfb8e105948403725dacd8c7aa5d53f56f6d69
Nov 02 03:50:07.255880 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.255849    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w" event={"ID":"4772f6cc-590c-4777-b7da-1b3310316f79","Type":"ContainerStarted","Data":"c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43"}
Nov 02 03:50:07.256079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.255903    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w" event={"ID":"4772f6cc-590c-4777-b7da-1b3310316f79","Type":"ContainerStarted","Data":"c0219e988bef25049ca9903715fcf87433bf5d1cc290d476312d5c0ffcde7348"}
Nov 02 03:50:07.256079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.255931    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:07.257570 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.257547    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9" event={"ID":"b225fb39-617c-4075-8b02-611f6528a3b3","Type":"ContainerStarted","Data":"ad93184ec91734fa752e967227f30bb05e47a406405cc104ef0df51ac7fe16b1"}
Nov 02 03:50:07.257700 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.257583    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9" event={"ID":"b225fb39-617c-4075-8b02-611f6528a3b3","Type":"ContainerStarted","Data":"1a31de01c626cb02c1c2d98870bfb8e105948403725dacd8c7aa5d53f56f6d69"}
Nov 02 03:50:07.257700 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.257613    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:07.372257 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.372162    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w" podStartSLOduration=1.372149522 podStartE2EDuration="1.372149522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:50:07.371114755 +0000 UTC m=+3751.166463319" watchObservedRunningTime="2026-11-02 03:50:07.372149522 +0000 UTC m=+3751.167498086"
Nov 02 03:50:07.494135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:07.493874    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9" podStartSLOduration=1.493852821 podStartE2EDuration="1.493852821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:50:07.486382833 +0000 UTC m=+3751.281731423" watchObservedRunningTime="2026-11-02 03:50:07.493852821 +0000 UTC m=+3751.289201387"
Nov 02 03:50:09.621652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:09.621606    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:13.065026 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:13.064997    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"]
Nov 02 03:50:13.115764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:13.115732    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9fd61eb2-6hk8c"]
Nov 02 03:50:14.352484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:14.352450    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ac82e3e-6052-47f0-adce-a5a845b6694b" path="/var/lib/kubelet/pods/3ac82e3e-6052-47f0-adce-a5a845b6694b/volumes"
Nov 02 03:50:14.606964 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:14.606265    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:15.245795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:15.245760    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:17.050226 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:17.050195    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"]
Nov 02 03:50:17.100073 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:17.100038    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-91858a07-phnqp"]
Nov 02 03:50:18.266946 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:18.266918    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:50:18.267367 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:18.266991    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:50:18.352202 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:18.352166    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b1a180c-29f9-4f05-8719-7af8d044425f" path="/var/lib/kubelet/pods/8b1a180c-29f9-4f05-8719-7af8d044425f/volumes"
Nov 02 03:50:19.479797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:19.479767    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:19.638595 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:19.638566    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:21.513432 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:21.513385    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:24.664076 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:24.664043    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:29.817152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:29.817114    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:34.880120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:34.880087    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:35.190075 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:50:35.189987    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:50:38.548771 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:38.548743    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:39.772159 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:39.772130    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:50:40.230942 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:40.230854    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:45.210685 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:45.210613    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:49.688562 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:49.688524    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:50.068956 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:50.068923    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:54.906977 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:54.906944    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:55.178756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:55.178675    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:50:59.984020 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:50:59.983980    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:04.760673 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:04.760599    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:04.821180 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:04.821146    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"]
Nov 02 03:51:04.821391 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:04.821351    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy" containerID="cri-o://f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968" gracePeriod=30
Nov 02 03:51:04.868614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:04.868580    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"]
Nov 02 03:51:04.869090 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:04.869056    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy" containerID="cri-o://5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a" gracePeriod=30
Nov 02 03:51:05.189271 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:51:05.189191    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:51:09.569521 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:09.569480    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:10.146025 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.145995    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:51:10.179687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.179656    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:51:10.298403 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298366    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-credential-socket\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.298571 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298430    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/05282817-c06b-4790-bb94-cb414c255936-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05282817-c06b-4790-bb94-cb414c255936-istio-podinfo\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.298571 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298484    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-certs\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.298571 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298527    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-socket\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.298819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298570    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-envoy\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.298819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298645    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-podinfo\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.298943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298826    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-istio-token\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-istio-token\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.298943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298858    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-socket\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.298943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298735    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-credential-socket" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.298943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298794    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-socket" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.298943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298846    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-certs" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.298943 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298903    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-token\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-token\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.299259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298952    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-kube-api-access-bpwz4\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-kube-api-access-bpwz4\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.299259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.298993    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-data\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.299259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299077    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-envoy\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.299259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299107    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-kube-api-access-rrvzv\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-kube-api-access-rrvzv\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.299259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299164    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05282817-c06b-4790-bb94-cb414c255936-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05282817-c06b-4790-bb94-cb414c255936-istiod-ca-cert\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.299259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299207    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-credential-socket\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.299259 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299257    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istiod-ca-cert\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.299693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299391    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-socket" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.299693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299446    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-data\") pod \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\" (UID: \"231e6700-1eb8-4578-ae18-bfbd14ae0b55\") "
Nov 02 03:51:10.299693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299519    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-certs\") pod \"05282817-c06b-4790-bb94-cb414c255936\" (UID: \"05282817-c06b-4790-bb94-cb414c255936\") "
Nov 02 03:51:10.300126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.299877    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-data" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.300227 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300154    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-credential-socket" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.300288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300266    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istiod-ca-cert" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:51:10.300440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300357    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-data" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.300440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300392    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05282817-c06b-4790-bb94-cb414c255936-istiod-ca-cert" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:51:10.300578 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300474    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-certs" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.300958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300889    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.300958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300921    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.300958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300936    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.300958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300952    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.300958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300968    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.300958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300984    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.300958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.300998    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.301507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.301012    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.301507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.301026    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.301507 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.301041    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05282817-c06b-4790-bb94-cb414c255936-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.301782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.301754    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-envoy" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.301916 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.301859    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05282817-c06b-4790-bb94-cb414c255936-istio-podinfo" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:51:10.302551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.302528    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-istio-token" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:51:10.302687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.302562    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-token" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:51:10.302789 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.302771    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-podinfo" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:51:10.302980 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.302961    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-kube-api-access-bpwz4" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "kube-api-access-bpwz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:51:10.303466 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.303442    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-kube-api-access-rrvzv" pod "231e6700-1eb8-4578-ae18-bfbd14ae0b55" (UID: "231e6700-1eb8-4578-ae18-bfbd14ae0b55"). InnerVolumeSpecName "kube-api-access-rrvzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:51:10.303813 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.303784    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-envoy" pod "05282817-c06b-4790-bb94-cb414c255936" (UID: "05282817-c06b-4790-bb94-cb414c255936"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:51:10.402208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.402179    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.402208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.402211    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.402381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.402221    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.402381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.402230    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.402381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.402240    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpwz4\" (UniqueName: \"kubernetes.io/projected/05282817-c06b-4790-bb94-cb414c255936-kube-api-access-bpwz4\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.402381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.402249    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05282817-c06b-4790-bb94-cb414c255936-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.402381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.402257    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrvzv\" (UniqueName: \"kubernetes.io/projected/231e6700-1eb8-4578-ae18-bfbd14ae0b55-kube-api-access-rrvzv\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.402381 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.402281    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05282817-c06b-4790-bb94-cb414c255936-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:51:10.630927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.630828    4407 generic.go:356] "Generic (PLEG): container finished" podID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerID="f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968" exitCode=0
Nov 02 03:51:10.630927 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.630897    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"
Nov 02 03:51:10.631486 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.630925    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" event={"ID":"231e6700-1eb8-4578-ae18-bfbd14ae0b55","Type":"ContainerDied","Data":"f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968"}
Nov 02 03:51:10.631486 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.630980    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz" event={"ID":"231e6700-1eb8-4578-ae18-bfbd14ae0b55","Type":"ContainerDied","Data":"11c1da744770e5dbf0fdb69e52bc080d5036c2fd2da3204ed6be31ba8877b46e"}
Nov 02 03:51:10.631486 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.630997    4407 scope.go:122] "RemoveContainer" containerID="f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968"
Nov 02 03:51:10.632519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.632492    4407 generic.go:356] "Generic (PLEG): container finished" podID="05282817-c06b-4790-bb94-cb414c255936" containerID="5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a" exitCode=0
Nov 02 03:51:10.632669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.632556    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"
Nov 02 03:51:10.632669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.632580    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" event={"ID":"05282817-c06b-4790-bb94-cb414c255936","Type":"ContainerDied","Data":"5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a"}
Nov 02 03:51:10.632782 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.632668    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7" event={"ID":"05282817-c06b-4790-bb94-cb414c255936","Type":"ContainerDied","Data":"d11d4ccc65b778279786dde4021002553d355ee52d57a8017522ca3982231d4e"}
Nov 02 03:51:10.641104 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.641084    4407 scope.go:122] "RemoveContainer" containerID="f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968"
Nov 02 03:51:10.641383 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:51:10.641356    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968\": container with ID starting with f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968 not found: ID does not exist" containerID="f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968"
Nov 02 03:51:10.641452 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.641391    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968"} err="failed to get container status \"f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968\": rpc error: code = NotFound desc = could not find container \"f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968\": container with ID starting with f1ffc70c51b8ac4935371ec39434ef55865d0bfc43ee0c9f2b82c9aa5f898968 not found: ID does not exist"
Nov 02 03:51:10.641452 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.641407    4407 scope.go:122] "RemoveContainer" containerID="5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a"
Nov 02 03:51:10.650552 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.650529    4407 scope.go:122] "RemoveContainer" containerID="5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a"
Nov 02 03:51:10.650951 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:51:10.650931    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a\": container with ID starting with 5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a not found: ID does not exist" containerID="5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a"
Nov 02 03:51:10.651044 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.650959    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a"} err="failed to get container status \"5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a\": rpc error: code = NotFound desc = could not find container \"5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a\": container with ID starting with 5676a808909f50d6d159c340e76d4997aeab903e6ee698027099dc9afe0e253a not found: ID does not exist"
Nov 02 03:51:10.797859 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.797825    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"]
Nov 02 03:51:10.848324 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.848300    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-94qlz"]
Nov 02 03:51:10.912440 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.912361    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"]
Nov 02 03:51:10.915871 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:10.915845    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-56vq7"]
Nov 02 03:51:12.351791 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:12.351755    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05282817-c06b-4790-bb94-cb414c255936" path="/var/lib/kubelet/pods/05282817-c06b-4790-bb94-cb414c255936/volumes"
Nov 02 03:51:12.352280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:12.352264    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" path="/var/lib/kubelet/pods/231e6700-1eb8-4578-ae18-bfbd14ae0b55/volumes"
Nov 02 03:51:14.310874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:14.310841    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:19.066837 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:19.066752    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:22.165010 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:22.164960    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:23.839903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:23.839867    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:23.998763 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:23.998728    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:24.146412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:24.146303    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:25.883729 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:25.883685    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:28.774976 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:28.774937    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:33.488575 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:33.488530    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:35.193263 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:51:35.193222    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:51:37.188014 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:37.187985    4407 scope.go:122] "RemoveContainer" containerID="ebab1f20942352026b6500496f09c450beb98b08457b8bb6286ea7061ef453e4"
Nov 02 03:51:37.200450 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:37.200432    4407 scope.go:122] "RemoveContainer" containerID="907b16f768528f777f621b1bd792c9e08217345ca94160ec414f3340dfdb68d9"
Nov 02 03:51:38.217775 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:38.217743    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:39.394538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:39.394506    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:51:43.018756 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:43.018718    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:46.587160 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:46.587129    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:47.836821 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:47.836767    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:50.039882 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:50.039847    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:52.630787 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:52.630757    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:51:57.503483 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:51:57.503451    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:02.474234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:02.474201    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:05.189465 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:52:05.189424    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:52:07.210652 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:07.210594    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:08.981734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:08.981701    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:52:09.710719 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:09.710684    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:11.956130 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:11.956096    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:15.758785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:15.758751    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:16.707069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:16.707041    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:21.483127 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:21.483094    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:26.302068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:26.302035    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:30.999589 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:30.999553    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:35.188273 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:52:35.188236    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:52:35.769512 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:35.769469    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:40.514909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:40.514876    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:42.025152 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:42.025120    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:42.177374 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:42.177343    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:45.287272 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:45.287214    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:50.095260 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:50.095178    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:50.249803 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:50.249771    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:50.410730 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:50.410641    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:50.858924 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:50.858890    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:51.755917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.755885    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"]
Nov 02 03:52:51.756484 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.756467    4407 cpu_manager.go:414] "containerMap: removing container" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy"
Nov 02 03:52:51.756551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.756486    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy"
Nov 02 03:52:51.756551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.756514    4407 cpu_manager.go:414] "containerMap: removing container" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy"
Nov 02 03:52:51.756551 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.756519    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy"
Nov 02 03:52:51.756667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.756601    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="05282817-c06b-4790-bb94-cb414c255936" containerName="istio-proxy"
Nov 02 03:52:51.756667 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.756614    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="231e6700-1eb8-4578-ae18-bfbd14ae0b55" containerName="istio-proxy"
Nov 02 03:52:51.760208 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.760188    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.797555 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.797525    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-model-cache\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.797793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.797773    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-home\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.797887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.797812    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tmp-dir\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.797957 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.797928    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgw4r\" (UniqueName: \"kubernetes.io/projected/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-kube-api-access-mgw4r\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.798023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.798006    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-dshm\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.798066 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.798050    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tls-certs\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.817008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.816975    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"]
Nov 02 03:52:51.862897 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.862861    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w4-ac7ab4\"/\"default-dockercfg-fsq9h\""
Nov 02 03:52:51.863068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.862908    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w4-ac7ab4\"/\"auth-test-model-ac7ab4-kserve-self-signed-certs\""
Nov 02 03:52:51.866613 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.866599    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w4-ac7ab4\"/\"kube-root-ca.crt\""
Nov 02 03:52:51.866673 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.866596    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w4-ac7ab4\"/\"openshift-service-ca.crt\""
Nov 02 03:52:51.899485 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.899447    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-model-cache\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.899761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.899524    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-home\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.899761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.899558    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tmp-dir\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.899761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.899618    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgw4r\" (UniqueName: \"kubernetes.io/projected/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-kube-api-access-mgw4r\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.899761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.899723    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-dshm\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.899761 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.899760    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tls-certs\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.911015 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.900956    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tmp-dir\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.911015 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.901252    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-model-cache\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.911015 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.910904    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-home\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.914328 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.914246    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:51.916561 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.916414    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tls-certs\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.917557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.917526    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-dshm\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:51.960644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:51.960600    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgw4r\" (UniqueName: \"kubernetes.io/projected/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-kube-api-access-mgw4r\") pod \"auth-test-model-ac7ab4-kserve-7667c87877-744hc\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") " pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:52.073032 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:52.072751    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:52.447071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:52.447044    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"]
Nov 02 03:52:52.447740 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:52:52.447716    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b595c06_42ea_45a5_bbdc_94eb8cd14c4c.slice/crio-2169c7b4a8b5aa457afa65eb78f86deceda97c8e62c8d0b1448eafd0dae57d7b WatchSource:0}: Error finding container 2169c7b4a8b5aa457afa65eb78f86deceda97c8e62c8d0b1448eafd0dae57d7b: Status 404 returned error can't find the container with id 2169c7b4a8b5aa457afa65eb78f86deceda97c8e62c8d0b1448eafd0dae57d7b
Nov 02 03:52:53.253525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:53.253483    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc" event={"ID":"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c","Type":"ContainerStarted","Data":"781e8a0a085cd6a45c243a968dab299b0d8ba1c4bb21eb54cc5b819168686d74"}
Nov 02 03:52:53.253525 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:53.253529    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc" event={"ID":"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c","Type":"ContainerStarted","Data":"2169c7b4a8b5aa457afa65eb78f86deceda97c8e62c8d0b1448eafd0dae57d7b"}
Nov 02 03:52:53.367968 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:53.367914    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc" podStartSLOduration=2.367900962 podStartE2EDuration="2.367900962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:52:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:52:53.367694609 +0000 UTC m=+3917.163043175" watchObservedRunningTime="2026-11-02 03:52:53.367900962 +0000 UTC m=+3917.163249527"
Nov 02 03:52:54.073778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:54.073747    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:52:55.070540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:55.070511    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:59.866865 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:59.866829    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:52:59.990760 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:52:59.990727    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:53:02.073235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:02.073205    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:53:02.085811 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:02.085791    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:53:02.318131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:02.318103    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:53:04.581982 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:04.581955    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:05.188952 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:53:05.188908    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:53:09.278402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:09.278371    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:14.055150 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:14.055119    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:18.807056 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:18.807017    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:23.744948 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:23.744907    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:28.471939 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:28.471906    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:30.329400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:30.329365    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:33.233465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:33.233429    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:35.190324 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:53:35.190278    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:53:37.947777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:37.947741    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:42.682374 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:42.682340    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:43.805842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:43.805805    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:53:46.740707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.740674    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"]
Nov 02 03:53:46.744540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.744523    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.802189 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.802156    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"]
Nov 02 03:53:46.809759 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.809732    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-model-cache\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.809921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.809765    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-home\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.809921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.809806    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-tmp-dir\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.810016 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.809993    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4cxr\" (UniqueName: \"kubernetes.io/projected/47905bc2-4636-48e9-ac4f-4908513b4034-kube-api-access-l4cxr\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.810059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.810028    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47905bc2-4636-48e9-ac4f-4908513b4034-tls-certs\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.810162 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.810149    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-dshm\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.848437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.848400    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-3d14ee\"/\"kube-root-ca.crt\""
Nov 02 03:53:46.848602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.848481    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-3d14ee\"/\"default-dockercfg-kb75p\""
Nov 02 03:53:46.848602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.848498    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-3d14ee\"/\"openshift-service-ca.crt\""
Nov 02 03:53:46.848602 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.848547    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-3d14ee\"/\"auth-test-model-3d14ee-kserve-self-signed-certs\""
Nov 02 03:53:46.867005 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.866970    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:46.911636 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.911579    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-model-cache\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.911847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.911660    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-home\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.911847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.911725    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-tmp-dir\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.911847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.911785    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4cxr\" (UniqueName: \"kubernetes.io/projected/47905bc2-4636-48e9-ac4f-4908513b4034-kube-api-access-l4cxr\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.911847 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.911812    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47905bc2-4636-48e9-ac4f-4908513b4034-tls-certs\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.912139 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.911907    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-dshm\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.912534 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.912509    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-tmp-dir\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.913405 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.913377    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-home\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.913577 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.913553    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-model-cache\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.914905 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.914880    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-dshm\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.915673 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.915607    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47905bc2-4636-48e9-ac4f-4908513b4034-tls-certs\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:46.970415 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:46.970387    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4cxr\" (UniqueName: \"kubernetes.io/projected/47905bc2-4636-48e9-ac4f-4908513b4034-kube-api-access-l4cxr\") pod \"auth-test-model-3d14ee-kserve-9546f5c9c-bhskb\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") " pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:47.056694 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:47.056658    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:47.434666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:47.433990    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"]
Nov 02 03:53:47.488887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:47.488863    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:47.602699 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:47.602663    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb" event={"ID":"47905bc2-4636-48e9-ac4f-4908513b4034","Type":"ContainerStarted","Data":"ee71ef9588903481106b57e909566a89dd89f85283953d24a459c2252184b5fd"}
Nov 02 03:53:47.602890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:47.602708    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb" event={"ID":"47905bc2-4636-48e9-ac4f-4908513b4034","Type":"ContainerStarted","Data":"f67b934323e460854dab9895c6bd8620f5562925d1414b1f637b867f23ac90b3"}
Nov 02 03:53:47.719609 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:47.719503    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb" podStartSLOduration=1.7194905440000001 podStartE2EDuration="1.719490544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:53:47.717387142 +0000 UTC m=+3971.512735707" watchObservedRunningTime="2026-11-02 03:53:47.719490544 +0000 UTC m=+3971.514839108"
Nov 02 03:53:48.056826 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:48.056791    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:50.737321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:50.737290    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:52.263687 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:52.263650    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:57.057706 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:57.057667    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:57.070778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:57.070750    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:53:57.174181 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:57.174149    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:53:57.676117 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:53:57.676089    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:54:01.901570 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:01.901524    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:05.191069 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:54:05.191031    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:54:06.625697 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:06.625667    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:09.374965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:09.374935    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:11.372339 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:11.372294    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:16.087377 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:16.087334    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:20.937325 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:20.937249    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:25.673400 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:25.673344    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:25.848563 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:25.848531    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:30.871386 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:30.871352    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:35.189606 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:54:35.189569    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:54:35.676164 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:35.676134    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:37.792594 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:37.792563    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"]
Nov 02 03:54:37.902075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:37.902045    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"]
Nov 02 03:54:38.172112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:38.172041    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:38.325672 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:38.325613    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:40.170919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:40.170886    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:54:40.467204 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:40.467122    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:42.793668 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:42.793600    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9" podUID="b225fb39-617c-4075-8b02-611f6528a3b3" containerName="payload-processing" containerID="cri-o://ad93184ec91734fa752e967227f30bb05e47a406405cc104ef0df51ac7fe16b1" gracePeriod=25
Nov 02 03:54:42.902845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:42.902807    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w" podUID="4772f6cc-590c-4777-b7da-1b3310316f79" containerName="payload-pre-processing" containerID="cri-o://c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43" gracePeriod=25
Nov 02 03:54:42.947220 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:42.947186    4407 generic.go:356] "Generic (PLEG): container finished" podID="b225fb39-617c-4075-8b02-611f6528a3b3" containerID="ad93184ec91734fa752e967227f30bb05e47a406405cc104ef0df51ac7fe16b1" exitCode=0
Nov 02 03:54:42.947418 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:42.947255    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9" event={"ID":"b225fb39-617c-4075-8b02-611f6528a3b3","Type":"ContainerDied","Data":"ad93184ec91734fa752e967227f30bb05e47a406405cc104ef0df51ac7fe16b1"}
Nov 02 03:54:43.088654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.088271    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:54:43.157790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.157763    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b225fb39-617c-4075-8b02-611f6528a3b3-kube-api-access-rrg9r\" (UniqueName: \"kubernetes.io/projected/b225fb39-617c-4075-8b02-611f6528a3b3-kube-api-access-rrg9r\") pod \"b225fb39-617c-4075-8b02-611f6528a3b3\" (UID: \"b225fb39-617c-4075-8b02-611f6528a3b3\") "
Nov 02 03:54:43.157921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.157882    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b225fb39-617c-4075-8b02-611f6528a3b3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b225fb39-617c-4075-8b02-611f6528a3b3-plugins-config-volume\") pod \"b225fb39-617c-4075-8b02-611f6528a3b3\" (UID: \"b225fb39-617c-4075-8b02-611f6528a3b3\") "
Nov 02 03:54:43.158281 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.158230    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b225fb39-617c-4075-8b02-611f6528a3b3-plugins-config-volume" pod "b225fb39-617c-4075-8b02-611f6528a3b3" (UID: "b225fb39-617c-4075-8b02-611f6528a3b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:54:43.158424 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.158404    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b225fb39-617c-4075-8b02-611f6528a3b3-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:54:43.160315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.160294    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b225fb39-617c-4075-8b02-611f6528a3b3-kube-api-access-rrg9r" pod "b225fb39-617c-4075-8b02-611f6528a3b3" (UID: "b225fb39-617c-4075-8b02-611f6528a3b3"). InnerVolumeSpecName "kube-api-access-rrg9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:54:43.179155 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.179136    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:54:43.259442 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.259414    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4772f6cc-590c-4777-b7da-1b3310316f79-kube-api-access-czjkd\" (UniqueName: \"kubernetes.io/projected/4772f6cc-590c-4777-b7da-1b3310316f79-kube-api-access-czjkd\") pod \"4772f6cc-590c-4777-b7da-1b3310316f79\" (UID: \"4772f6cc-590c-4777-b7da-1b3310316f79\") "
Nov 02 03:54:43.259591 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.259507    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4772f6cc-590c-4777-b7da-1b3310316f79-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4772f6cc-590c-4777-b7da-1b3310316f79-plugins-config-volume\") pod \"4772f6cc-590c-4777-b7da-1b3310316f79\" (UID: \"4772f6cc-590c-4777-b7da-1b3310316f79\") "
Nov 02 03:54:43.259914 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.259899    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrg9r\" (UniqueName: \"kubernetes.io/projected/b225fb39-617c-4075-8b02-611f6528a3b3-kube-api-access-rrg9r\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:54:43.259965 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.259906    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4772f6cc-590c-4777-b7da-1b3310316f79-plugins-config-volume" pod "4772f6cc-590c-4777-b7da-1b3310316f79" (UID: "4772f6cc-590c-4777-b7da-1b3310316f79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:54:43.261751 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.261731    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4772f6cc-590c-4777-b7da-1b3310316f79-kube-api-access-czjkd" pod "4772f6cc-590c-4777-b7da-1b3310316f79" (UID: "4772f6cc-590c-4777-b7da-1b3310316f79"). InnerVolumeSpecName "kube-api-access-czjkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:54:43.360909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.360819    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czjkd\" (UniqueName: \"kubernetes.io/projected/4772f6cc-590c-4777-b7da-1b3310316f79-kube-api-access-czjkd\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:54:43.360909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.360850    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4772f6cc-590c-4777-b7da-1b3310316f79-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:54:43.955434 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.955397    4407 generic.go:356] "Generic (PLEG): container finished" podID="4772f6cc-590c-4777-b7da-1b3310316f79" containerID="c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43" exitCode=0
Nov 02 03:54:43.955917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.955452    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"
Nov 02 03:54:43.955917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.955473    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w" event={"ID":"4772f6cc-590c-4777-b7da-1b3310316f79","Type":"ContainerDied","Data":"c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43"}
Nov 02 03:54:43.955917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.955527    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w" event={"ID":"4772f6cc-590c-4777-b7da-1b3310316f79","Type":"ContainerDied","Data":"c0219e988bef25049ca9903715fcf87433bf5d1cc290d476312d5c0ffcde7348"}
Nov 02 03:54:43.955917 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.955548    4407 scope.go:122] "RemoveContainer" containerID="c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43"
Nov 02 03:54:43.957414 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.957395    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9" event={"ID":"b225fb39-617c-4075-8b02-611f6528a3b3","Type":"ContainerDied","Data":"1a31de01c626cb02c1c2d98870bfb8e105948403725dacd8c7aa5d53f56f6d69"}
Nov 02 03:54:43.957527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.957427    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"
Nov 02 03:54:43.967578 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.967557    4407 scope.go:122] "RemoveContainer" containerID="c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43"
Nov 02 03:54:43.967924 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:54:43.967904    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43\": container with ID starting with c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43 not found: ID does not exist" containerID="c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43"
Nov 02 03:54:43.968059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.967938    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43"} err="failed to get container status \"c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43\": rpc error: code = NotFound desc = could not find container \"c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43\": container with ID starting with c0365ad8de74cf9580bc2f19cfee8b5fdcb6d0a99e45d35c41a6ba169ec86a43 not found: ID does not exist"
Nov 02 03:54:43.968059 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:43.967963    4407 scope.go:122] "RemoveContainer" containerID="ad93184ec91734fa752e967227f30bb05e47a406405cc104ef0df51ac7fe16b1"
Nov 02 03:54:44.122426 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:44.122340    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"]
Nov 02 03:54:44.125379 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:44.125361    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-86f4c4-6d49ccf78c-m6gv9"]
Nov 02 03:54:44.236133 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:44.236099    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"]
Nov 02 03:54:44.241698 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:44.241672    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-86f4c4-65798bb56c-8824w"]
Nov 02 03:54:44.351913 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:44.351885    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4772f6cc-590c-4777-b7da-1b3310316f79" path="/var/lib/kubelet/pods/4772f6cc-590c-4777-b7da-1b3310316f79/volumes"
Nov 02 03:54:44.352313 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:44.352299    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b225fb39-617c-4075-8b02-611f6528a3b3" path="/var/lib/kubelet/pods/b225fb39-617c-4075-8b02-611f6528a3b3/volumes"
Nov 02 03:54:45.301811 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:45.301772    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:50.079239 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:50.079207    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:51.103298 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:51.103266    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:54.885743 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:54.885705    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:54:59.674770 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:54:59.674735    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:04.406778 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:04.406748    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:05.189035 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:55:05.188999    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:55:09.164121 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:09.164089    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:13.916001 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:13.915970    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:14.065873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:14.065840    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:18.667082 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:18.667046    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:23.434222 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:23.434189    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:27.576212 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:27.576178    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:29.119038 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:29.119001    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:29.335649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:29.335598    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:29.488887 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:29.488818    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:30.131439 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:30.131408    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:34.427902 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:34.427863    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:35.189194 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:55:35.189151    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:55:35.754051 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:35.753966    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:35.913958 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:35.913931    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:36.083888 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:36.083803    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:36.630020 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:36.629988    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:36.914809 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:36.914721    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"]
Nov 02 03:55:36.915274 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:36.914983    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" containerID="cri-o://d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08" gracePeriod=30
Nov 02 03:55:37.087970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.087942    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"]
Nov 02 03:55:37.088504 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.088489    4407 cpu_manager.go:414] "containerMap: removing container" podUID="4772f6cc-590c-4777-b7da-1b3310316f79" containerName="payload-pre-processing"
Nov 02 03:55:37.088549 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.088506    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4772f6cc-590c-4777-b7da-1b3310316f79" containerName="payload-pre-processing"
Nov 02 03:55:37.088549 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.088522    4407 cpu_manager.go:414] "containerMap: removing container" podUID="b225fb39-617c-4075-8b02-611f6528a3b3" containerName="payload-processing"
Nov 02 03:55:37.088549 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.088527    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b225fb39-617c-4075-8b02-611f6528a3b3" containerName="payload-processing"
Nov 02 03:55:37.088672 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.088651    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="4772f6cc-590c-4777-b7da-1b3310316f79" containerName="payload-pre-processing"
Nov 02 03:55:37.088672 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.088663    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="b225fb39-617c-4075-8b02-611f6528a3b3" containerName="payload-processing"
Nov 02 03:55:37.092035 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.092019    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.187757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.187157    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-00359a-openshift-default-dockercfg-rpghn\""
Nov 02 03:55:37.193684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.190423    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"]
Nov 02 03:55:37.194399 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194293    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-token\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.194399 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194372    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-podinfo\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.194591 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194515    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-envoy\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.194858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194608    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-data\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.194858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194688    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-certs\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.194858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194716    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-socket\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.194858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194739    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0afac7-116a-4f7d-b1f6-06802d052807-istiod-ca-cert\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.194858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194766    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l77bk\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-kube-api-access-l77bk\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.194858 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.194831    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-credential-socket\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.212318 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.212127    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:37.295666 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.295610    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-token\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.295783 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.295715    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-podinfo\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.295826 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.295790    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-envoy\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.295882 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.295865    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-data\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.295988 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.295921    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-certs\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296012    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-socket\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296049 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296041    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0afac7-116a-4f7d-b1f6-06802d052807-istiod-ca-cert\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296071    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l77bk\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-kube-api-access-l77bk\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296156 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296137    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-credential-socket\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296294    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-certs\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296344    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-data\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296488 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296473    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-credential-socket\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296755 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296732    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0afac7-116a-4f7d-b1f6-06802d052807-istiod-ca-cert\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.296864 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.296788    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-socket\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.298827 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.298804    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-envoy\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.299069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.299051    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-podinfo\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.354921 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.354893    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-token\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.355092 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.355065    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l77bk\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-kube-api-access-l77bk\") pod \"e2e-noten-00359a-openshift-default-6764dcd565-4l58q\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") " pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.368897 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.368865    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:37.405069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.405038    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:37.830442 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.830411    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"]
Nov 02 03:55:37.830683 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:55:37.830661    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0afac7_116a_4f7d_b1f6_06802d052807.slice/crio-e3260c5866866fb664b7d694f37e306298dee76cd6db90f92d873519fceaf3f9 WatchSource:0}: Error finding container e3260c5866866fb664b7d694f37e306298dee76cd6db90f92d873519fceaf3f9: Status 404 returned error can't find the container with id e3260c5866866fb664b7d694f37e306298dee76cd6db90f92d873519fceaf3f9
Nov 02 03:55:37.833308 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.833273    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:55:37.833367 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.833344    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:55:37.833404 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:37.833373    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:55:38.294908 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:38.294875    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q" event={"ID":"0f0afac7-116a-4f7d-b1f6-06802d052807","Type":"ContainerStarted","Data":"ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5"}
Nov 02 03:55:38.295374 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:38.294931    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q" event={"ID":"0f0afac7-116a-4f7d-b1f6-06802d052807","Type":"ContainerStarted","Data":"e3260c5866866fb664b7d694f37e306298dee76cd6db90f92d873519fceaf3f9"}
Nov 02 03:55:38.406123 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:38.406091    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:38.411508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:38.411483    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:38.412396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:38.412357    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q" podStartSLOduration=1.412345554 podStartE2EDuration="1.412345554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:55:38.411754175 +0000 UTC m=+4082.207102739" watchObservedRunningTime="2026-11-02 03:55:38.412345554 +0000 UTC m=+4082.207694120"
Nov 02 03:55:38.819492 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:38.819454    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:39.066177 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:39.066144    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:39.229155 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:39.229072    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:39.301680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:39.301655    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:39.302794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:39.302774    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:55:39.407736 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:39.407706    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:39.410785 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:39.410758    4407 patch_prober.go:29] interesting pod/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:55:39.410898 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:39.410814    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:55:42.193558 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.193486    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:55:42.258330 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258298    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-kube-api-access-cpwdp\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-kube-api-access-cpwdp\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258392    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istiod-ca-cert\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258432    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-data\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258499 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258449    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-credential-socket\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258679 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258513    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-certs\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258679 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258565    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-socket\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258679 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258614    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-podinfo\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258708    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-envoy\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258777    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-data" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:55:42.258852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258790    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-certs" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:55:42.258852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258823    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-token\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-token\") pod \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\" (UID: \"2ae2f898-f465-4757-bb66-7e0d51a7b06f\") "
Nov 02 03:55:42.258852 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258805    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-credential-socket" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:55:42.259098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258852    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-socket" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:55:42.259098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.258904    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istiod-ca-cert" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:55:42.259430 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.259409    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.259538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.259433    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.259538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.259448    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.259538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.259460    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.259538 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.259474    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.260780 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.260753    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-kube-api-access-cpwdp" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "kube-api-access-cpwdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:55:42.260892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.260856    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-token" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:55:42.261478 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.261454    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-envoy" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:55:42.261478 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.261459    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-podinfo" pod "2ae2f898-f465-4757-bb66-7e0d51a7b06f" (UID: "2ae2f898-f465-4757-bb66-7e0d51a7b06f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:55:42.320970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.320924    4407 generic.go:356] "Generic (PLEG): container finished" podID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerID="d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08" exitCode=0
Nov 02 03:55:42.320970 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.320975    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"
Nov 02 03:55:42.321235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.320985    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" event={"ID":"2ae2f898-f465-4757-bb66-7e0d51a7b06f","Type":"ContainerDied","Data":"d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08"}
Nov 02 03:55:42.321235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.321058    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm" event={"ID":"2ae2f898-f465-4757-bb66-7e0d51a7b06f","Type":"ContainerDied","Data":"a4bd18e9ceba944413c5b007483dec63134414ce7a2e955594e57edb3eae63da"}
Nov 02 03:55:42.321235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.321088    4407 scope.go:122] "RemoveContainer" containerID="d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08"
Nov 02 03:55:42.333465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.333446    4407 scope.go:122] "RemoveContainer" containerID="d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08"
Nov 02 03:55:42.333797 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:55:42.333778    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08\": container with ID starting with d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08 not found: ID does not exist" containerID="d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08"
Nov 02 03:55:42.333872 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.333808    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08"} err="failed to get container status \"d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08\": rpc error: code = NotFound desc = could not find container \"d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08\": container with ID starting with d6dd265738aff2c1a9a388f0435073647b8e7e6e6bcfd67a5e79ad9fada7cc08 not found: ID does not exist"
Nov 02 03:55:42.361142 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.361113    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.361296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.361138    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.361296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.361161    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.361296 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.361171    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpwdp\" (UniqueName: \"kubernetes.io/projected/2ae2f898-f465-4757-bb66-7e0d51a7b06f-kube-api-access-cpwdp\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:55:42.525315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.525274    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"]
Nov 02 03:55:42.527876 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.527850    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:55:42.530745 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:42.530725    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5ac93f-openshift-default-77d76bbd85-lwwpm"]
Nov 02 03:55:43.687788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:43.687757    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:44.091187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:44.091157    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:44.355720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:44.355616    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" path="/var/lib/kubelet/pods/2ae2f898-f465-4757-bb66-7e0d51a7b06f/volumes"
Nov 02 03:55:47.260891 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:47.260863    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:47.427949 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:47.427919    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:48.877331 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:48.877229    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:51.412539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:51.412501    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:53.630757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:53.630721    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:58.339593 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:58.339554    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:55:58.887908 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:55:58.887876    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:03.185315 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:03.185288    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:05.189301 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:56:05.189264    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:56:07.981651 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:07.981594    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:12.750502 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:12.750467    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:17.471644 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:17.471598    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:19.359731 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:19.359687    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:56:21.254351 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:21.254314    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"]
Nov 02 03:56:21.254794 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:21.254514    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q" podUID="0f0afac7-116a-4f7d-b1f6-06802d052807" containerName="istio-proxy" containerID="cri-o://ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5" gracePeriod=30
Nov 02 03:56:21.270842 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:21.270811    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:21.578554 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:21.578466    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:21.798654 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:21.798605    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:22.236330 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:22.236296    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:24.304406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:24.304372    4407 patch_prober.go:29] interesting pod/e2e-noten-00359a-openshift-default-6764dcd565-4l58q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 03:56:24.304845 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:24.304432    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q" podUID="0f0afac7-116a-4f7d-b1f6-06802d052807" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 03:56:26.542098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.542074    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:56:26.579893 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.579856    4407 generic.go:356] "Generic (PLEG): container finished" podID="0f0afac7-116a-4f7d-b1f6-06802d052807" containerID="ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5" exitCode=0
Nov 02 03:56:26.580069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.579898    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"
Nov 02 03:56:26.580069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.579936    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q" event={"ID":"0f0afac7-116a-4f7d-b1f6-06802d052807","Type":"ContainerDied","Data":"ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5"}
Nov 02 03:56:26.580069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.580004    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q" event={"ID":"0f0afac7-116a-4f7d-b1f6-06802d052807","Type":"ContainerDied","Data":"e3260c5866866fb664b7d694f37e306298dee76cd6db90f92d873519fceaf3f9"}
Nov 02 03:56:26.580069 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.580033    4407 scope.go:122] "RemoveContainer" containerID="ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5"
Nov 02 03:56:26.592884 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.592865    4407 scope.go:122] "RemoveContainer" containerID="ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5"
Nov 02 03:56:26.593231 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:56:26.593196    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5\": container with ID starting with ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5 not found: ID does not exist" containerID="ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5"
Nov 02 03:56:26.593350 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.593238    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5"} err="failed to get container status \"ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5\": rpc error: code = NotFound desc = could not find container \"ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5\": container with ID starting with ea598f98b9929c293001e89c88c117b9f2e39db4d95bbe4e720f7db08af003b5 not found: ID does not exist"
Nov 02 03:56:26.714566 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714532    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-socket\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.714793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714588    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-kube-api-access-l77bk\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-kube-api-access-l77bk\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.714793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714650    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f0afac7-116a-4f7d-b1f6-06802d052807-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0afac7-116a-4f7d-b1f6-06802d052807-istiod-ca-cert\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.714793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714681    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-token\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-token\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.714793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714724    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-podinfo\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.714793 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714751    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-envoy\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.715080 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714812    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-credential-socket\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.715080 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714928    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-data\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.715080 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.714929    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-socket" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:56:26.715080 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.715071    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-certs\") pod \"0f0afac7-116a-4f7d-b1f6-06802d052807\" (UID: \"0f0afac7-116a-4f7d-b1f6-06802d052807\") "
Nov 02 03:56:26.715303 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.715160    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-data" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:56:26.715711 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.715689    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.715890 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.715875    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.716009 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.715784    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0afac7-116a-4f7d-b1f6-06802d052807-istiod-ca-cert" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 03:56:26.716129 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.715833    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-certs" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:56:26.716129 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.716027    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-credential-socket" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:56:26.718402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.718094    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-kube-api-access-l77bk" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "kube-api-access-l77bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:56:26.718402 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.718194    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-envoy" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:56:26.718659 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.718611    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-podinfo" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 03:56:26.718758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.718742    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-token" pod "0f0afac7-116a-4f7d-b1f6-06802d052807" (UID: "0f0afac7-116a-4f7d-b1f6-06802d052807"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:56:26.732703 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.732681    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:26.817683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.817602    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0afac7-116a-4f7d-b1f6-06802d052807-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.817683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.817658    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.817683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.817669    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.817683 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.817679    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.817936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.817689    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.817936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.817696    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0afac7-116a-4f7d-b1f6-06802d052807-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.817936 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.817705    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l77bk\" (UniqueName: \"kubernetes.io/projected/0f0afac7-116a-4f7d-b1f6-06802d052807-kube-api-access-l77bk\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:56:26.965790 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:26.965752    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:27.006283 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:27.006252    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"]
Nov 02 03:56:27.008892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:27.008864    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-00359a-openshift-default-6764dcd565-4l58q"]
Nov 02 03:56:28.352631 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:28.352588    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f0afac7-116a-4f7d-b1f6-06802d052807" path="/var/lib/kubelet/pods/0f0afac7-116a-4f7d-b1f6-06802d052807/volumes"
Nov 02 03:56:31.765528 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:31.765487    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:35.189442 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:56:35.189408    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:56:36.550541 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:36.545601    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:41.251302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:41.251264    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:46.107972 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:46.107942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:50.845310 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:50.845276    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:51.769125 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:51.769096    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:55.572510 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:55.572467    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:56.050419 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:56.050390    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:56:58.995183 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:56:58.995147    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:00.476764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:00.476734    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:03.685288 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:03.685254    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:05.189359 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:57:05.189319    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:57:05.280983 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:05.280958    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:10.073017 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:10.072984    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:14.810871 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:14.810832    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:19.677505 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:19.677427    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:20.539483 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:20.539453    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:57:22.616027 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:22.615987    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:23.628902 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:23.628858    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:23.836919 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:23.836873    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:24.595217 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:24.595189    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:27.530539 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:27.530502    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:27.707169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:27.707124    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:27.870332 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:27.870253    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:28.932598 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:28.932566    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:29.712967 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:29.712935    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:30.932130 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:30.932097    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:31.692377 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.692344    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"]
Nov 02 03:57:31.693068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.693047    4407 cpu_manager.go:414] "containerMap: removing container" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy"
Nov 02 03:57:31.693068 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.693067    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy"
Nov 02 03:57:31.693250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.693081    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0f0afac7-116a-4f7d-b1f6-06802d052807" containerName="istio-proxy"
Nov 02 03:57:31.693250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.693089    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f0afac7-116a-4f7d-b1f6-06802d052807" containerName="istio-proxy"
Nov 02 03:57:31.693250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.693220    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ae2f898-f465-4757-bb66-7e0d51a7b06f" containerName="istio-proxy"
Nov 02 03:57:31.693250 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.693232    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f0afac7-116a-4f7d-b1f6-06802d052807" containerName="istio-proxy"
Nov 02 03:57:31.696824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.696803    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.749726 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.749696    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-9b7f43-openshift-default-dockercfg-fcgjz\""
Nov 02 03:57:31.779559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.779530    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-token\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.779779 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.779592    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-credential-socket\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.779779 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.779745    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-data\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.779907 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.779814    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp8w7\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-kube-api-access-zp8w7\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.779907 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.779867    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60105b4e-1c8b-4fbd-b501-5608e1392a55-istiod-ca-cert\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.779999 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.779976    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-socket\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.780035 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.780015    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-podinfo\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.780089 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.780038    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-envoy\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.780161 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.780141    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-certs\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.805039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.805011    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"]
Nov 02 03:57:31.812846 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.812818    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:31.881291 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881253    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-certs\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.881490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881333    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-token\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.881490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881396    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-credential-socket\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.881490 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881468    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-data\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.881767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881537    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp8w7\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-kube-api-access-zp8w7\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.881767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881600    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60105b4e-1c8b-4fbd-b501-5608e1392a55-istiod-ca-cert\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.881767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881696    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-certs\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.881767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881726    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-socket\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.881767 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881747    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-credential-socket\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.882023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881783    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-podinfo\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.882023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881818    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-envoy\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.882023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.881815    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-data\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.882023 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.882003    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-socket\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.882254 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.882231    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60105b4e-1c8b-4fbd-b501-5608e1392a55-istiod-ca-cert\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.884271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.884240    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-envoy\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.884449 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.884430    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-podinfo\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.938749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.938701    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-token\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:31.939095 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:31.938909    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp8w7\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-kube-api-access-zp8w7\") pod \"e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") " pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:32.010582 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:32.010547    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:32.046634 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:32.046573    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:32.445749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:32.445719    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"]
Nov 02 03:57:32.446164 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:57:32.446143    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60105b4e_1c8b_4fbd_b501_5608e1392a55.slice/crio-60a16c6b38f7b073489fd1b2b7f77f0a34a2e442663522bf3ab1ab60887a055a WatchSource:0}: Error finding container 60a16c6b38f7b073489fd1b2b7f77f0a34a2e442663522bf3ab1ab60887a055a: Status 404 returned error can't find the container with id 60a16c6b38f7b073489fd1b2b7f77f0a34a2e442663522bf3ab1ab60887a055a
Nov 02 03:57:32.448680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:32.448647    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:57:32.448757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:32.448729    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:57:32.448797 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:32.448778    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:57:32.973851 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:32.973809    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" event={"ID":"60105b4e-1c8b-4fbd-b501-5608e1392a55","Type":"ContainerStarted","Data":"faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65"}
Nov 02 03:57:32.974334 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:32.973899    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" event={"ID":"60105b4e-1c8b-4fbd-b501-5608e1392a55","Type":"ContainerStarted","Data":"60a16c6b38f7b073489fd1b2b7f77f0a34a2e442663522bf3ab1ab60887a055a"}
Nov 02 03:57:33.011168 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:33.011136    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:33.017126 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:33.017103    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:33.092079 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:33.092022    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podStartSLOduration=2.092009666 podStartE2EDuration="2.092009666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:57:33.090831529 +0000 UTC m=+4196.886180095" watchObservedRunningTime="2026-11-02 03:57:33.092009666 +0000 UTC m=+4196.887358231"
Nov 02 03:57:33.980403 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:33.980370    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:33.981242 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:33.981226    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:57:34.230194 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:34.230165    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:34.402342 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:34.402308    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:34.685493 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:34.685410    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:35.189268 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:57:35.189231    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:57:35.849000 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:35.848970    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:36.030516 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:36.030482    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:36.206681 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:36.206566    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:39.046601 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.046571    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:39.208940 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.208887    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:39.592749 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.589824    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"]
Nov 02 03:57:39.594731 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.594706    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:39.651027 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.650996    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"]
Nov 02 03:57:39.693383 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.693348    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d01ef-6350-4f20-a5a6-b8e95afdae9b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r\" (UID: \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:39.693567 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.693552    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mtkq\" (UniqueName: \"kubernetes.io/projected/766d01ef-6350-4f20-a5a6-b8e95afdae9b-kube-api-access-4mtkq\") pod \"payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r\" (UID: \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:39.697989 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.697966    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-9b7f43\""
Nov 02 03:57:39.698098 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.698042    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-9b7f43-dockercfg-jzs2x\""
Nov 02 03:57:39.795135 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.795097    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d01ef-6350-4f20-a5a6-b8e95afdae9b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r\" (UID: \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:39.795321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.795285    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mtkq\" (UniqueName: \"kubernetes.io/projected/766d01ef-6350-4f20-a5a6-b8e95afdae9b-kube-api-access-4mtkq\") pod \"payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r\" (UID: \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:39.795819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.795799    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d01ef-6350-4f20-a5a6-b8e95afdae9b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r\" (UID: \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:39.852997 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.852920    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mtkq\" (UniqueName: \"kubernetes.io/projected/766d01ef-6350-4f20-a5a6-b8e95afdae9b-kube-api-access-4mtkq\") pod \"payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r\" (UID: \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:39.868795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.868767    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:39.910060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:39.910029    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:40.286039 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:40.286012    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"]
Nov 02 03:57:40.286522 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:57:40.286500    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod766d01ef_6350_4f20_a5a6_b8e95afdae9b.slice/crio-deb73e12d74b015068fc93b3f1c97fcb9fd5588d7c4cb0b14800dc6ea86b9c52 WatchSource:0}: Error finding container deb73e12d74b015068fc93b3f1c97fcb9fd5588d7c4cb0b14800dc6ea86b9c52: Status 404 returned error can't find the container with id deb73e12d74b015068fc93b3f1c97fcb9fd5588d7c4cb0b14800dc6ea86b9c52
Nov 02 03:57:41.025386 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:41.025351    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r" event={"ID":"766d01ef-6350-4f20-a5a6-b8e95afdae9b","Type":"ContainerStarted","Data":"b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159"}
Nov 02 03:57:41.025581 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:41.025405    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r" event={"ID":"766d01ef-6350-4f20-a5a6-b8e95afdae9b","Type":"ContainerStarted","Data":"deb73e12d74b015068fc93b3f1c97fcb9fd5588d7c4cb0b14800dc6ea86b9c52"}
Nov 02 03:57:41.025581 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:41.025474    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:41.140660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:41.140596    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r" podStartSLOduration=2.140581716 podStartE2EDuration="2.140581716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:57:41.138843672 +0000 UTC m=+4204.934192238" watchObservedRunningTime="2026-11-02 03:57:41.140581716 +0000 UTC m=+4204.935930281"
Nov 02 03:57:41.748835 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:41.748801    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:43.341886 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:43.341851    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:43.498234 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:43.498199    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:44.669171 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:44.669133    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:49.467764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:49.467731    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:52.034985 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:52.034956    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 03:57:52.098559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:52.098524    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:52.651819 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:52.651783    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:54.278684 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:54.278653    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:57:59.059174 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:57:59.059145    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:00.935540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:00.935506    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:01.547695 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:01.547657    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:01.740945 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:01.740910    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:03.864992 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:03.864942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:05.189508 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:58:05.189471    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:58:05.868075 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:05.868042    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:06.042333 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:06.042293    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:06.053508 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:06.053482    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"]
Nov 02 03:58:07.294060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:07.294024    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:08.913664 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:08.913633    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:11.414272 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.414237    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:11.576008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.575979    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:11.800064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.800031    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"]
Nov 02 03:58:11.804370 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.804347    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:11.860116 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.860082    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"]
Nov 02 03:58:11.938064 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.938021    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zxgf\" (UniqueName: \"kubernetes.io/projected/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-kube-api-access-7zxgf\") pod \"payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv\" (UID: \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:11.952688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.944744    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv\" (UID: \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:11.952688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.945252    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-ae1315-dockercfg-2dr5s\""
Nov 02 03:58:11.952688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:11.945304    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-ae1315\""
Nov 02 03:58:12.046306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:12.046270    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zxgf\" (UniqueName: \"kubernetes.io/projected/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-kube-api-access-7zxgf\") pod \"payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv\" (UID: \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:12.046519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:12.046406    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv\" (UID: \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:12.047134 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:12.047110    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv\" (UID: \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:12.103873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:12.103808    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zxgf\" (UniqueName: \"kubernetes.io/projected/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-kube-api-access-7zxgf\") pod \"payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv\" (UID: \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:12.118739 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:12.118715    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:12.492764 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:12.492735    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"]
Nov 02 03:58:12.493939 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:58:12.493916    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a05c6c_45d8_49bf_b6dc_4f77e40a131b.slice/crio-52640206cdef8ef654e7c66b7d2a1394419c3fb3c21deb1cdbfc16805ad8e366 WatchSource:0}: Error finding container 52640206cdef8ef654e7c66b7d2a1394419c3fb3c21deb1cdbfc16805ad8e366: Status 404 returned error can't find the container with id 52640206cdef8ef654e7c66b7d2a1394419c3fb3c21deb1cdbfc16805ad8e366
Nov 02 03:58:13.266649 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:13.266602    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv" event={"ID":"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b","Type":"ContainerStarted","Data":"8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c"}
Nov 02 03:58:13.266849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:13.266669    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv" event={"ID":"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b","Type":"ContainerStarted","Data":"52640206cdef8ef654e7c66b7d2a1394419c3fb3c21deb1cdbfc16805ad8e366"}
Nov 02 03:58:13.266849 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:13.266690    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:13.381061 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:13.381005    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv" podStartSLOduration=2.38099141 podStartE2EDuration="2.38099141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:58:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:58:13.380661475 +0000 UTC m=+4237.176010039" watchObservedRunningTime="2026-11-02 03:58:13.38099141 +0000 UTC m=+4237.176339975"
Nov 02 03:58:13.696933 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:13.696853    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:18.469926 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:18.469892    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:19.284284 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:19.284249    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:58:21.074909 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.074839    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc" podUID="6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" containerName="main" containerID="cri-o://781e8a0a085cd6a45c243a968dab299b0d8ba1c4bb21eb54cc5b819168686d74" gracePeriod=45
Nov 02 03:58:21.329169 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.329136    4407 generic.go:356] "Generic (PLEG): container finished" podID="6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" containerID="781e8a0a085cd6a45c243a968dab299b0d8ba1c4bb21eb54cc5b819168686d74" exitCode=0
Nov 02 03:58:21.329302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.329233    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc" event={"ID":"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c","Type":"ContainerDied","Data":"781e8a0a085cd6a45c243a968dab299b0d8ba1c4bb21eb54cc5b819168686d74"}
Nov 02 03:58:21.329302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.329268    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc" event={"ID":"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c","Type":"ContainerDied","Data":"2169c7b4a8b5aa457afa65eb78f86deceda97c8e62c8d0b1448eafd0dae57d7b"}
Nov 02 03:58:21.329302 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.329285    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2169c7b4a8b5aa457afa65eb78f86deceda97c8e62c8d0b1448eafd0dae57d7b"
Nov 02 03:58:21.348702 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.348679    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:58:21.476277 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.476239    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-model-cache\") pod \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") "
Nov 02 03:58:21.476473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.476348    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-kube-api-access-mgw4r\" (UniqueName: \"kubernetes.io/projected/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-kube-api-access-mgw4r\") pod \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") "
Nov 02 03:58:21.476473 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.476419    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-dshm\") pod \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") "
Nov 02 03:58:21.476603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.476497    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-model-cache" pod "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" (UID: "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:58:21.476603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.476529    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tmp-dir\") pod \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") "
Nov 02 03:58:21.476603 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.476568    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-home\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-home\") pod \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") "
Nov 02 03:58:21.476830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.476646    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tls-certs\" (UniqueName: \"kubernetes.io/secret/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tls-certs\") pod \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\" (UID: \"6b595c06-42ea-45a5-bbdc-94eb8cd14c4c\") "
Nov 02 03:58:21.476830 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.476781    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tmp-dir" pod "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" (UID: "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:58:21.477311 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.477284    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-home" pod "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" (UID: "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:58:21.477496 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.477371    4407 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-model-cache\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:58:21.477680 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.477664    4407 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tmp-dir\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:58:21.479008 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.478982    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-kube-api-access-mgw4r" pod "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" (UID: "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c"). InnerVolumeSpecName "kube-api-access-mgw4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:58:21.479228 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.479208    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-dshm" pod "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" (UID: "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:58:21.479708 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.479687    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tls-certs" pod "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" (UID: "6b595c06-42ea-45a5-bbdc-94eb8cd14c4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:58:21.578804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.578726    4407 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-home\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:58:21.578804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.578753    4407 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-tls-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:58:21.578804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.578765    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgw4r\" (UniqueName: \"kubernetes.io/projected/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-kube-api-access-mgw4r\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:58:21.578804 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:21.578775    4407 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c-dshm\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:58:22.307601 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:22.307557    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc" podUID="6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.127:8000/ready\": context deadline exceeded"
Nov 02 03:58:22.335037 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:22.335008    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"
Nov 02 03:58:22.500597 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:22.500567    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"]
Nov 02 03:58:22.504146 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:22.504125    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ac7ab4/auth-test-model-ac7ab4-kserve-7667c87877-744hc"]
Nov 02 03:58:23.328607 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:23.328572    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:24.275901 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:24.275870    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 03:58:24.351823 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:24.351796    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" path="/var/lib/kubelet/pods/6b595c06-42ea-45a5-bbdc-94eb8cd14c4c/volumes"
Nov 02 03:58:28.120108 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:28.120075    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:28.327127 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:28.327093    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:32.927550 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:32.927518    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:33.207459 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:33.207371    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:35.190407 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:58:35.190372    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:58:37.764462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:37.764431    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:37.964301 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:37.964269    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:38.425280 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.425242    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"]
Nov 02 03:58:38.425949 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.425928    4407 cpu_manager.go:414] "containerMap: removing container" podUID="6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" containerName="main"
Nov 02 03:58:38.425949 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.425949    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" containerName="main"
Nov 02 03:58:38.426131 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.426071    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b595c06-42ea-45a5-bbdc-94eb8cd14c4c" containerName="main"
Nov 02 03:58:38.430816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.430795    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496058    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-envoy\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496248 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496125    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb3623e-4997-4231-9052-54b65d6441f1-istio-podinfo\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496309 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496283    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-socket\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496388    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bkjk\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-kube-api-access-5bkjk\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496474 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496456    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb3623e-4997-4231-9052-54b65d6441f1-istiod-ca-cert\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496537 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496526    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-credential-socket\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496660 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496638    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-istio-token\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496720 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496703    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-data\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.496774 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.496760    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-certs\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.521757 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.521733    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-61f3b6-openshift-default-dockercfg-r5dn9\""
Nov 02 03:58:38.539036 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.539009    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"]
Nov 02 03:58:38.590612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.590579    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:38.597903 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.597879    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bkjk\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-kube-api-access-5bkjk\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598085 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.597916    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb3623e-4997-4231-9052-54b65d6441f1-istiod-ca-cert\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598085 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.597969    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-credential-socket\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598085 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598006    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-istio-token\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598085 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598025    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-data\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598085 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598070    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-certs\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598351 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598113    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-envoy\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598351 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598213    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb3623e-4997-4231-9052-54b65d6441f1-istio-podinfo\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598351 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598292    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-socket\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598446    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-credential-socket\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598511 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598495    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-data\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598592 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598534    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-certs\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598740 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598719    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-socket\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.598788 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.598725    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb3623e-4997-4231-9052-54b65d6441f1-istiod-ca-cert\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.600501 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.600482    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-envoy\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.600862 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.600841    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb3623e-4997-4231-9052-54b65d6441f1-istio-podinfo\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.675693 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.675493    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-istio-token\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.685469 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.683330    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bkjk\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-kube-api-access-5bkjk\") pod \"e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") " pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.744321 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.744276    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:38.752910 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:38.752879    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:39.172795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:39.172764    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"]
Nov 02 03:58:39.173150 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:58:39.173083    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb3623e_4997_4231_9052_54b65d6441f1.slice/crio-0e1d8a9a1f01225684f7a4454a8fb7576635ab5ac72315820fc29ff8da828f99 WatchSource:0}: Error finding container 0e1d8a9a1f01225684f7a4454a8fb7576635ab5ac72315820fc29ff8da828f99: Status 404 returned error can't find the container with id 0e1d8a9a1f01225684f7a4454a8fb7576635ab5ac72315820fc29ff8da828f99
Nov 02 03:58:39.175824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:39.175789    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:58:39.175915 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:39.175855    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:58:39.175915 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:39.175883    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:58:39.442584 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:39.442487    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" event={"ID":"dfb3623e-4997-4231-9052-54b65d6441f1","Type":"ContainerStarted","Data":"8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d"}
Nov 02 03:58:39.442584 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:39.442553    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" event={"ID":"dfb3623e-4997-4231-9052-54b65d6441f1","Type":"ContainerStarted","Data":"0e1d8a9a1f01225684f7a4454a8fb7576635ab5ac72315820fc29ff8da828f99"}
Nov 02 03:58:39.561959 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:39.561899    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" podStartSLOduration=1.561885953 podStartE2EDuration="1.561885953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:58:39.559251749 +0000 UTC m=+4263.354600315" watchObservedRunningTime="2026-11-02 03:58:39.561885953 +0000 UTC m=+4263.357234518"
Nov 02 03:58:39.744647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:39.744609    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:40.750691 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:40.750662    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:41.455349 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:41.455320    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:41.456437 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:41.456415    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 03:58:42.595221 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:42.595191    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:42.765261 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:42.765233    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:42.924034 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:42.923947    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:45.838744 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:45.838713    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:45.996763 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:45.996737    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:46.384257 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.384224    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"]
Nov 02 03:58:46.388185 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.388166    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:46.444406 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.444373    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"]
Nov 02 03:58:46.491273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.491247    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-61f3b6\""
Nov 02 03:58:46.491273 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.491249    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-61f3b6-dockercfg-bv8qh\""
Nov 02 03:58:46.507557 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.507531    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sz7q\" (UniqueName: \"kubernetes.io/projected/27fed099-7c7c-4417-9192-4229fac3a586-kube-api-access-6sz7q\") pod \"payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w\" (UID: \"27fed099-7c7c-4417-9192-4229fac3a586\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:46.507707 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.507693    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27fed099-7c7c-4417-9192-4229fac3a586-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w\" (UID: \"27fed099-7c7c-4417-9192-4229fac3a586\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:46.608671 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.608605    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sz7q\" (UniqueName: \"kubernetes.io/projected/27fed099-7c7c-4417-9192-4229fac3a586-kube-api-access-6sz7q\") pod \"payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w\" (UID: \"27fed099-7c7c-4417-9192-4229fac3a586\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:46.608892 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.608780    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27fed099-7c7c-4417-9192-4229fac3a586-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w\" (UID: \"27fed099-7c7c-4417-9192-4229fac3a586\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:46.609464 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.609444    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27fed099-7c7c-4417-9192-4229fac3a586-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w\" (UID: \"27fed099-7c7c-4417-9192-4229fac3a586\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:46.666734 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.666668    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sz7q\" (UniqueName: \"kubernetes.io/projected/27fed099-7c7c-4417-9192-4229fac3a586-kube-api-access-6sz7q\") pod \"payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w\" (UID: \"27fed099-7c7c-4417-9192-4229fac3a586\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:46.702269 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:46.702242    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:47.075605 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:47.075556    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"]
Nov 02 03:58:47.076030 ip-10-0-131-112 kubenswrapper[4407]: W1102 03:58:47.076007    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27fed099_7c7c_4417_9192_4229fac3a586.slice/crio-28b2691c4f801b040b98f75df3b5081d2e52b04a91832979a3a6f2206381eaed WatchSource:0}: Error finding container 28b2691c4f801b040b98f75df3b5081d2e52b04a91832979a3a6f2206381eaed: Status 404 returned error can't find the container with id 28b2691c4f801b040b98f75df3b5081d2e52b04a91832979a3a6f2206381eaed
Nov 02 03:58:47.492669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:47.492561    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w" event={"ID":"27fed099-7c7c-4417-9192-4229fac3a586","Type":"ContainerStarted","Data":"a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50"}
Nov 02 03:58:47.492669 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:47.492646    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w" event={"ID":"27fed099-7c7c-4417-9192-4229fac3a586","Type":"ContainerStarted","Data":"28b2691c4f801b040b98f75df3b5081d2e52b04a91832979a3a6f2206381eaed"}
Nov 02 03:58:47.492874 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:47.492685    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:58:47.618812 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:47.618757    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w" podStartSLOduration=1.618742348 podStartE2EDuration="1.618742348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 03:58:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 03:58:47.607222195 +0000 UTC m=+4271.402570763" watchObservedRunningTime="2026-11-02 03:58:47.618742348 +0000 UTC m=+4271.414090913"
Nov 02 03:58:47.903869 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:47.903843    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:48.114559 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:48.114465    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:48.264500 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:48.264455    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:52.442099 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:52.442067    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:53.110360 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:53.110324    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:53.580647 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:53.580587    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:57.926129 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:57.926091    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:58.092465 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:58.092436    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:58.245384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:58.245354    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:58:58.510218 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:58:58.510155    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 03:59:02.864504 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:02.864471    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:05.189121 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:59:05.189084    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:59:07.618832 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:07.618762    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:12.392263 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:12.392226    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:17.198462 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:17.198432    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:17.957612 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:17.957575    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:18.536235 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:18.536194    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:18.712141 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:18.712067    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:19.990300 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:19.990251    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:20.139138 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:20.139103    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:22.199994 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:22.199954    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:22.371037 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:22.371006    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:22.532112 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:22.532073    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:25.854614 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:25.854570    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:26.027608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:26.027574    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:27.525944 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:27.525904    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:27.687187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:27.687155    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"]
Nov 02 03:59:27.717836 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:27.717797    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:32.488107 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:32.488072    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:33.951362 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:33.951320    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 03:59:35.190012 ip-10-0-131-112 kubenswrapper[4407]: E1102 03:59:35.189903    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 03:59:37.243881 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:37.243846    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:37.324412 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:37.324379    4407 scope.go:122] "RemoveContainer" containerID="781e8a0a085cd6a45c243a968dab299b0d8ba1c4bb21eb54cc5b819168686d74"
Nov 02 03:59:37.426810 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:37.426778    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:37.602275 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:37.602183    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:42.351855 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:42.351822    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:42.708060 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:42.707952    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb" podUID="47905bc2-4636-48e9-ac4f-4908513b4034" containerName="main" containerID="cri-o://ee71ef9588903481106b57e909566a89dd89f85283953d24a459c2252184b5fd" gracePeriod=45
Nov 02 03:59:42.865777 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:42.865748    4407 generic.go:356] "Generic (PLEG): container finished" podID="47905bc2-4636-48e9-ac4f-4908513b4034" containerID="ee71ef9588903481106b57e909566a89dd89f85283953d24a459c2252184b5fd" exitCode=0
Nov 02 03:59:42.865974 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:42.865827    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb" event={"ID":"47905bc2-4636-48e9-ac4f-4908513b4034","Type":"ContainerDied","Data":"ee71ef9588903481106b57e909566a89dd89f85283953d24a459c2252184b5fd"}
Nov 02 03:59:42.978187 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:42.978119    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:59:43.102143 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.102105    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-tmp-dir\") pod \"47905bc2-4636-48e9-ac4f-4908513b4034\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") "
Nov 02 03:59:43.102329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.102231    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-dshm\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-dshm\") pod \"47905bc2-4636-48e9-ac4f-4908513b4034\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") "
Nov 02 03:59:43.102329 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.102302    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-home\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-home\") pod \"47905bc2-4636-48e9-ac4f-4908513b4034\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") "
Nov 02 03:59:43.102455 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.102329    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47905bc2-4636-48e9-ac4f-4908513b4034-kube-api-access-l4cxr\" (UniqueName: \"kubernetes.io/projected/47905bc2-4636-48e9-ac4f-4908513b4034-kube-api-access-l4cxr\") pod \"47905bc2-4636-48e9-ac4f-4908513b4034\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") "
Nov 02 03:59:43.102510 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.102493    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47905bc2-4636-48e9-ac4f-4908513b4034-tls-certs\" (UniqueName: \"kubernetes.io/secret/47905bc2-4636-48e9-ac4f-4908513b4034-tls-certs\") pod \"47905bc2-4636-48e9-ac4f-4908513b4034\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") "
Nov 02 03:59:43.102565 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.102526    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-model-cache\") pod \"47905bc2-4636-48e9-ac4f-4908513b4034\" (UID: \"47905bc2-4636-48e9-ac4f-4908513b4034\") "
Nov 02 03:59:43.102735 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.102710    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-tmp-dir" pod "47905bc2-4636-48e9-ac4f-4908513b4034" (UID: "47905bc2-4636-48e9-ac4f-4908513b4034"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:59:43.102938 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.102919    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-model-cache" pod "47905bc2-4636-48e9-ac4f-4908513b4034" (UID: "47905bc2-4636-48e9-ac4f-4908513b4034"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:59:43.103182 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.103166    4407 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-model-cache\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:59:43.103264 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.103189    4407 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-tmp-dir\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:59:43.103418 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.103395    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-home" pod "47905bc2-4636-48e9-ac4f-4908513b4034" (UID: "47905bc2-4636-48e9-ac4f-4908513b4034"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:59:43.106497 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.106466    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-dshm" pod "47905bc2-4636-48e9-ac4f-4908513b4034" (UID: "47905bc2-4636-48e9-ac4f-4908513b4034"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 03:59:43.106608 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.106522    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47905bc2-4636-48e9-ac4f-4908513b4034-kube-api-access-l4cxr" pod "47905bc2-4636-48e9-ac4f-4908513b4034" (UID: "47905bc2-4636-48e9-ac4f-4908513b4034"). InnerVolumeSpecName "kube-api-access-l4cxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 03:59:43.107758 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.107731    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47905bc2-4636-48e9-ac4f-4908513b4034-tls-certs" pod "47905bc2-4636-48e9-ac4f-4908513b4034" (UID: "47905bc2-4636-48e9-ac4f-4908513b4034"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 03:59:43.204091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.204059    4407 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-dshm\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:59:43.204091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.204086    4407 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47905bc2-4636-48e9-ac4f-4908513b4034-home\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:59:43.204091 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.204096    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4cxr\" (UniqueName: \"kubernetes.io/projected/47905bc2-4636-48e9-ac4f-4908513b4034-kube-api-access-l4cxr\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:59:43.204323 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.204105    4407 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47905bc2-4636-48e9-ac4f-4908513b4034-tls-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 03:59:43.872688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.872653    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"
Nov 02 03:59:43.872688 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.872676    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb" event={"ID":"47905bc2-4636-48e9-ac4f-4908513b4034","Type":"ContainerDied","Data":"f67b934323e460854dab9895c6bd8620f5562925d1414b1f637b867f23ac90b3"}
Nov 02 03:59:43.873247 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:43.872729    4407 scope.go:122] "RemoveContainer" containerID="ee71ef9588903481106b57e909566a89dd89f85283953d24a459c2252184b5fd"
Nov 02 03:59:44.044540 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.044506    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"]
Nov 02 03:59:44.095358 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.095323    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3d14ee/auth-test-model-3d14ee-kserve-9546f5c9c-bhskb"]
Nov 02 03:59:44.356816 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.356785    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47905bc2-4636-48e9-ac4f-4908513b4034" path="/var/lib/kubelet/pods/47905bc2-4636-48e9-ac4f-4908513b4034/volumes"
Nov 02 03:59:44.879586 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.879551    4407 generic.go:356] "Generic (PLEG): container finished" podID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerID="faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65" exitCode=0
Nov 02 03:59:44.880071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.879643    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" event={"ID":"60105b4e-1c8b-4fbd-b501-5608e1392a55","Type":"ContainerDied","Data":"faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65"}
Nov 02 03:59:44.880155 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.880135    4407 scope.go:122] "RemoveContainer" containerID="faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65"
Nov 02 03:59:44.881851 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.881825    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:59:44.881922 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.881896    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:59:44.881962 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:44.881927    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 03:59:45.896655 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:45.896085    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" event={"ID":"60105b4e-1c8b-4fbd-b501-5608e1392a55","Type":"ContainerStarted","Data":"70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358"}
Nov 02 03:59:46.011120 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:46.011085    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:59:46.012527 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:46.012506    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 02 03:59:46.012610 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:46.012551    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 02 03:59:47.011290 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:47.011254    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 02 03:59:47.011824 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:47.011309    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 02 03:59:47.146823 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:47.146787    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:48.011346 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:48.011308    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 02 03:59:48.011795 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:48.011365    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 02 03:59:48.980519 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:48.980488    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:59:49.011407 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:49.011366    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 02 03:59:49.011873 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:49.011423    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 02 03:59:50.011071 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:50.011033    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 02 03:59:50.011271 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:50.011098    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 02 03:59:51.011269 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:51.011235    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 02 03:59:51.011692 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:51.011286    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 02 03:59:51.969306 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:51.969266    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:52.020384 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:52.020353    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:59:52.021396 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:52.021376    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 03:59:52.790150 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:52.790110    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 03:59:56.805004 ip-10-0-131-112 kubenswrapper[4407]: I1102 03:59:56.804968    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:00.196777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.196740    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"]
Nov 02 04:00:00.197681 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.197662    4407 cpu_manager.go:414] "containerMap: removing container" podUID="47905bc2-4636-48e9-ac4f-4908513b4034" containerName="main"
Nov 02 04:00:00.197740 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.197684    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47905bc2-4636-48e9-ac4f-4908513b4034" containerName="main"
Nov 02 04:00:00.197893 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.197883    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="47905bc2-4636-48e9-ac4f-4908513b4034" containerName="main"
Nov 02 04:00:00.201847 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.201824    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"]
Nov 02 04:00:00.202000 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.201964    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:00.205763 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.205743    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"]
Nov 02 04:00:00.205869 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.205845    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:00.209598 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.209571    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:00.237841 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.237808    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h\" (UID: \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:00.237992 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.237876    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pkpb\" (UniqueName: \"kubernetes.io/projected/17485476-25e7-44cf-81d1-a485ae79f1ec-kube-api-access-5pkpb\") pod \"maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l\" (UID: \"17485476-25e7-44cf-81d1-a485ae79f1ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:00.238036 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.238016    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8xn8\" (UniqueName: \"kubernetes.io/projected/7116adfc-c3b3-47f5-949d-148d52c37b5a-kube-api-access-j8xn8\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v\" (UID: \"7116adfc-c3b3-47f5-949d-148d52c37b5a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:00.238093 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.238079    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tk42\" (UniqueName: \"kubernetes.io/projected/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-kube-api-access-8tk42\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h\" (UID: \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:00.238188 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.238171    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17485476-25e7-44cf-81d1-a485ae79f1ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l\" (UID: \"17485476-25e7-44cf-81d1-a485ae79f1ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:00.238238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.238224    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7116adfc-c3b3-47f5-949d-148d52c37b5a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v\" (UID: \"7116adfc-c3b3-47f5-949d-148d52c37b5a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:00.247466 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.247432    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"]
Nov 02 04:00:00.253252 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.253190    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"]
Nov 02 04:00:00.253384 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.253317    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:00.257439 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.257416    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"]
Nov 02 04:00:00.257612 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.257518    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:00.261391 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.261368    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"]
Nov 02 04:00:00.261523 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.261476    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:00.265375 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.265355    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:00.318945 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.318912    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"]
Nov 02 04:00:00.319223 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.319208    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"]
Nov 02 04:00:00.327016 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.326986    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9z5bk\""
Nov 02 04:00:00.339964 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.339918    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h8n9\" (UniqueName: \"kubernetes.io/projected/d09e8458-9745-49e1-8011-187ca69a56f1-kube-api-access-8h8n9\") pod \"maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn\" (UID: \"d09e8458-9745-49e1-8011-187ca69a56f1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:00.340133 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.339983    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pkpb\" (UniqueName: \"kubernetes.io/projected/17485476-25e7-44cf-81d1-a485ae79f1ec-kube-api-access-5pkpb\") pod \"maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l\" (UID: \"17485476-25e7-44cf-81d1-a485ae79f1ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:00.340133 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340079    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pdb6\" (UniqueName: \"kubernetes.io/projected/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-kube-api-access-9pdb6\") pod \"maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2\" (UID: \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:00.340237 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340128    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvm29\" (UniqueName: \"kubernetes.io/projected/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-kube-api-access-rvm29\") pod \"maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45\" (UID: \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:00.340353 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340333    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8xn8\" (UniqueName: \"kubernetes.io/projected/7116adfc-c3b3-47f5-949d-148d52c37b5a-kube-api-access-j8xn8\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v\" (UID: \"7116adfc-c3b3-47f5-949d-148d52c37b5a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:00.340403 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340384    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlcwr\" (UniqueName: \"kubernetes.io/projected/0cb04331-7309-471d-af6f-617e00bac3ce-kube-api-access-hlcwr\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv\" (UID: \"0cb04331-7309-471d-af6f-617e00bac3ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:00.340516 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340497    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tk42\" (UniqueName: \"kubernetes.io/projected/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-kube-api-access-8tk42\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h\" (UID: \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:00.340566 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340540    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45\" (UID: \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:00.340614 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340566    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17485476-25e7-44cf-81d1-a485ae79f1ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l\" (UID: \"17485476-25e7-44cf-81d1-a485ae79f1ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:00.340704 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340686    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7116adfc-c3b3-47f5-949d-148d52c37b5a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v\" (UID: \"7116adfc-c3b3-47f5-949d-148d52c37b5a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:00.340766 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340751    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2\" (UID: \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:00.340826 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340810    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb04331-7309-471d-af6f-617e00bac3ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv\" (UID: \"0cb04331-7309-471d-af6f-617e00bac3ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:00.340874 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340851    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h\" (UID: \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:00.340917 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.340876    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09e8458-9745-49e1-8011-187ca69a56f1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn\" (UID: \"d09e8458-9745-49e1-8011-187ca69a56f1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:00.341270 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.341253    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17485476-25e7-44cf-81d1-a485ae79f1ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l\" (UID: \"17485476-25e7-44cf-81d1-a485ae79f1ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:00.341470 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.341449    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7116adfc-c3b3-47f5-949d-148d52c37b5a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v\" (UID: \"7116adfc-c3b3-47f5-949d-148d52c37b5a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:00.341614 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.341592    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h\" (UID: \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:00.365079 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.365045    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"]
Nov 02 04:00:00.369253 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.369213    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"]
Nov 02 04:00:00.372798 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.372774    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"]
Nov 02 04:00:00.392249 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.392212    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"]
Nov 02 04:00:00.402292 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.402265    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pkpb\" (UniqueName: \"kubernetes.io/projected/17485476-25e7-44cf-81d1-a485ae79f1ec-kube-api-access-5pkpb\") pod \"maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l\" (UID: \"17485476-25e7-44cf-81d1-a485ae79f1ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:00.402573 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.402546    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8xn8\" (UniqueName: \"kubernetes.io/projected/7116adfc-c3b3-47f5-949d-148d52c37b5a-kube-api-access-j8xn8\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v\" (UID: \"7116adfc-c3b3-47f5-949d-148d52c37b5a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:00.402782 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.402762    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tk42\" (UniqueName: \"kubernetes.io/projected/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-kube-api-access-8tk42\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h\" (UID: \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:00.429317 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.429285    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"]
Nov 02 04:00:00.441772 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.441740    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45\" (UID: \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:00.441954 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.441852    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2\" (UID: \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:00.441954 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.441890    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb04331-7309-471d-af6f-617e00bac3ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv\" (UID: \"0cb04331-7309-471d-af6f-617e00bac3ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:00.441954 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.441939    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09e8458-9745-49e1-8011-187ca69a56f1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn\" (UID: \"d09e8458-9745-49e1-8011-187ca69a56f1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:00.442483 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.442459    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45\" (UID: \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:00.442693 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.442673    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2\" (UID: \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:00.442746 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.442673    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb04331-7309-471d-af6f-617e00bac3ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv\" (UID: \"0cb04331-7309-471d-af6f-617e00bac3ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:00.442746 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.442732    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h8n9\" (UniqueName: \"kubernetes.io/projected/d09e8458-9745-49e1-8011-187ca69a56f1-kube-api-access-8h8n9\") pod \"maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn\" (UID: \"d09e8458-9745-49e1-8011-187ca69a56f1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:00.442813 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.442776    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09e8458-9745-49e1-8011-187ca69a56f1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn\" (UID: \"d09e8458-9745-49e1-8011-187ca69a56f1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:00.442848 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.442808    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pdb6\" (UniqueName: \"kubernetes.io/projected/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-kube-api-access-9pdb6\") pod \"maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2\" (UID: \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:00.442887 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.442855    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvm29\" (UniqueName: \"kubernetes.io/projected/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-kube-api-access-rvm29\") pod \"maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45\" (UID: \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:00.443029 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.443008    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlcwr\" (UniqueName: \"kubernetes.io/projected/0cb04331-7309-471d-af6f-617e00bac3ce-kube-api-access-hlcwr\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv\" (UID: \"0cb04331-7309-471d-af6f-617e00bac3ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:00.501302 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.501268    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvm29\" (UniqueName: \"kubernetes.io/projected/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-kube-api-access-rvm29\") pod \"maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45\" (UID: \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:00.501486 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.501380    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pdb6\" (UniqueName: \"kubernetes.io/projected/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-kube-api-access-9pdb6\") pod \"maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2\" (UID: \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:00.501486 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.501415    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h8n9\" (UniqueName: \"kubernetes.io/projected/d09e8458-9745-49e1-8011-187ca69a56f1-kube-api-access-8h8n9\") pod \"maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn\" (UID: \"d09e8458-9745-49e1-8011-187ca69a56f1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:00.501486 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.501462    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlcwr\" (UniqueName: \"kubernetes.io/projected/0cb04331-7309-471d-af6f-617e00bac3ce-kube-api-access-hlcwr\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv\" (UID: \"0cb04331-7309-471d-af6f-617e00bac3ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:00.517066 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.517036    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:00.525951 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.525925    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:00.532030 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.532002    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:00.570140 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.568979    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:00.577812 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.576867    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:00.594035 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.587759    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:00.594035 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.591020    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:00.909058 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.909035    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"]
Nov 02 04:00:00.909514 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:00:00.909485    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17485476_25e7_44cf_81d1_a485ae79f1ec.slice/crio-3b41a6157da79d80902c77ab13da3852d26f5bfe537238d0033aff2afe9df56a WatchSource:0}: Error finding container 3b41a6157da79d80902c77ab13da3852d26f5bfe537238d0033aff2afe9df56a: Status 404 returned error can't find the container with id 3b41a6157da79d80902c77ab13da3852d26f5bfe537238d0033aff2afe9df56a
Nov 02 04:00:00.957820 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:00:00.957793    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7116adfc_c3b3_47f5_949d_148d52c37b5a.slice/crio-f2550cdec51d15de05143674ae2a48efc1082dd524de61f195042be7bfc6ba07 WatchSource:0}: Error finding container f2550cdec51d15de05143674ae2a48efc1082dd524de61f195042be7bfc6ba07: Status 404 returned error can't find the container with id f2550cdec51d15de05143674ae2a48efc1082dd524de61f195042be7bfc6ba07
Nov 02 04:00:00.958440 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.958412    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"]
Nov 02 04:00:00.960138 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:00:00.960111    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332b6b2c_fd0a_4f2b_96b0_0f6e23531182.slice/crio-a767d3f1114d0a1f8db2ff9acf5b40644e8f9c72a1c63b10091619ee08947311 WatchSource:0}: Error finding container a767d3f1114d0a1f8db2ff9acf5b40644e8f9c72a1c63b10091619ee08947311: Status 404 returned error can't find the container with id a767d3f1114d0a1f8db2ff9acf5b40644e8f9c72a1c63b10091619ee08947311
Nov 02 04:00:00.962500 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.962444    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"]
Nov 02 04:00:00.996077 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.996044    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" event={"ID":"17485476-25e7-44cf-81d1-a485ae79f1ec","Type":"ContainerStarted","Data":"3b41a6157da79d80902c77ab13da3852d26f5bfe537238d0033aff2afe9df56a"}
Nov 02 04:00:00.997674 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.997611    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" event={"ID":"332b6b2c-fd0a-4f2b-96b0-0f6e23531182","Type":"ContainerStarted","Data":"a767d3f1114d0a1f8db2ff9acf5b40644e8f9c72a1c63b10091619ee08947311"}
Nov 02 04:00:00.999223 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:00.999191    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" event={"ID":"7116adfc-c3b3-47f5-949d-148d52c37b5a","Type":"ContainerStarted","Data":"f2550cdec51d15de05143674ae2a48efc1082dd524de61f195042be7bfc6ba07"}
Nov 02 04:00:01.008470 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:01.008443    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"]
Nov 02 04:00:01.056871 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:01.056843    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"]
Nov 02 04:00:01.060756 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:01.060714    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"]
Nov 02 04:00:01.063359 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:01.063334    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"]
Nov 02 04:00:01.084906 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:00:01.084867    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb04331_7309_471d_af6f_617e00bac3ce.slice/crio-ae1c2e9bfec09146025e86fd97ffff89e34a7b2df2c7c1b2d0ce84b6d42330c6 WatchSource:0}: Error finding container ae1c2e9bfec09146025e86fd97ffff89e34a7b2df2c7c1b2d0ce84b6d42330c6: Status 404 returned error can't find the container with id ae1c2e9bfec09146025e86fd97ffff89e34a7b2df2c7c1b2d0ce84b6d42330c6
Nov 02 04:00:01.189003 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:00:01.188973    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6c1b51_5a41_4c74_b01d_5bdc57d662ac.slice/crio-afe9bc7380006b54ec887046b5bbe4579a2aa2a289827bff1ed6ca811201637c WatchSource:0}: Error finding container afe9bc7380006b54ec887046b5bbe4579a2aa2a289827bff1ed6ca811201637c: Status 404 returned error can't find the container with id afe9bc7380006b54ec887046b5bbe4579a2aa2a289827bff1ed6ca811201637c
Nov 02 04:00:01.189354 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:00:01.189308    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6bbf98_5906_4d5d_8a00_70390c2b2c44.slice/crio-8c84f8f768cd723edfdd4e0d423eda566332b244a640ffbd99e86ca156aabeb1 WatchSource:0}: Error finding container 8c84f8f768cd723edfdd4e0d423eda566332b244a640ffbd99e86ca156aabeb1: Status 404 returned error can't find the container with id 8c84f8f768cd723edfdd4e0d423eda566332b244a640ffbd99e86ca156aabeb1
Nov 02 04:00:01.661474 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:01.661440    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:02.007684 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.007654    4407 generic.go:356] "Generic (PLEG): container finished" podID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerID="e8932c1666803eca43c88b841bd4770f200287dd82640001feddf5fa29525a50" exitCode=60
Nov 02 04:00:02.007842 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.007717    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" event={"ID":"7116adfc-c3b3-47f5-949d-148d52c37b5a","Type":"ContainerDied","Data":"e8932c1666803eca43c88b841bd4770f200287dd82640001feddf5fa29525a50"}
Nov 02 04:00:02.008044 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.008030    4407 scope.go:122] "RemoveContainer" containerID="e8932c1666803eca43c88b841bd4770f200287dd82640001feddf5fa29525a50"
Nov 02 04:00:02.009542 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.009518    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" event={"ID":"17485476-25e7-44cf-81d1-a485ae79f1ec","Type":"ContainerStarted","Data":"d25af384404e6732873e3a270e794e08e1587dcc140f7fc1b77c682089428941"}
Nov 02 04:00:02.011371 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.011205    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" event={"ID":"332b6b2c-fd0a-4f2b-96b0-0f6e23531182","Type":"ContainerStarted","Data":"ee927dd8a07fbc0f277f0b46e81a07f64c4d8ee03bb4d0ac4a62f7ac5c7cde0e"}
Nov 02 04:00:02.012809 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.012785    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" event={"ID":"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac","Type":"ContainerStarted","Data":"18c5c7b3ea84fceddf41d2705fd52564a7bb74b3192322f20ef6e754856aa22b"}
Nov 02 04:00:02.012915 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.012834    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" event={"ID":"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac","Type":"ContainerStarted","Data":"afe9bc7380006b54ec887046b5bbe4579a2aa2a289827bff1ed6ca811201637c"}
Nov 02 04:00:02.014535 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.014512    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" event={"ID":"cc6bbf98-5906-4d5d-8a00-70390c2b2c44","Type":"ContainerStarted","Data":"801c17dc773a23f9afd39fbe734fa0da3ce3746b1fe6ba622ef765e7fe2eb3f0"}
Nov 02 04:00:02.014655 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.014558    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" event={"ID":"cc6bbf98-5906-4d5d-8a00-70390c2b2c44","Type":"ContainerStarted","Data":"8c84f8f768cd723edfdd4e0d423eda566332b244a640ffbd99e86ca156aabeb1"}
Nov 02 04:00:02.016086 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.016066    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" event={"ID":"d09e8458-9745-49e1-8011-187ca69a56f1","Type":"ContainerStarted","Data":"4311d8657e2e5e0641a512d007efe37b6efc026d7df8e675fe411765e6df5ddc"}
Nov 02 04:00:02.016182 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.016108    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" event={"ID":"d09e8458-9745-49e1-8011-187ca69a56f1","Type":"ContainerStarted","Data":"bb43c622f577e5e1c2af97e26e9c10cd703532d4a59316720919768fe8306829"}
Nov 02 04:00:02.017596 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.017574    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" event={"ID":"0cb04331-7309-471d-af6f-617e00bac3ce","Type":"ContainerStarted","Data":"24abfb7b2172771ef9175dc7f724e65ff19ce869950273a5e4a43a8cdcf2aae5"}
Nov 02 04:00:02.017696 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.017635    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" event={"ID":"0cb04331-7309-471d-af6f-617e00bac3ce","Type":"ContainerStarted","Data":"ae1c2e9bfec09146025e86fd97ffff89e34a7b2df2c7c1b2d0ce84b6d42330c6"}
Nov 02 04:00:02.239195 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.239094    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" podStartSLOduration=2.239074065 podStartE2EDuration="2.239074065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:00:02.236382655 +0000 UTC m=+4346.031731221" watchObservedRunningTime="2026-11-02 04:00:02.239074065 +0000 UTC m=+4346.034422635"
Nov 02 04:00:02.354010 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.353965    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" podStartSLOduration=2.353951895 podStartE2EDuration="2.353951895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:00:02.352214534 +0000 UTC m=+4346.147563098" watchObservedRunningTime="2026-11-02 04:00:02.353951895 +0000 UTC m=+4346.149300460"
Nov 02 04:00:02.469897 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.469842    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" podStartSLOduration=2.469829494 podStartE2EDuration="2.469829494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:00:02.467081232 +0000 UTC m=+4346.262429797" watchObservedRunningTime="2026-11-02 04:00:02.469829494 +0000 UTC m=+4346.265178058"
Nov 02 04:00:02.588526 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.588478    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" podStartSLOduration=2.5884637919999998 podStartE2EDuration="2.588463792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:00:02.584700699 +0000 UTC m=+4346.380049265" watchObservedRunningTime="2026-11-02 04:00:02.588463792 +0000 UTC m=+4346.383812358"
Nov 02 04:00:02.702397 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.702345    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" podStartSLOduration=2.7023319089999998 podStartE2EDuration="2.702331909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:00:02.700254143 +0000 UTC m=+4346.495602707" watchObservedRunningTime="2026-11-02 04:00:02.702331909 +0000 UTC m=+4346.497680473"
Nov 02 04:00:02.818451 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:02.818349    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" podStartSLOduration=2.8183311 podStartE2EDuration="2.8183311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:00:02.815779731 +0000 UTC m=+4346.611128297" watchObservedRunningTime="2026-11-02 04:00:02.8183311 +0000 UTC m=+4346.613679666"
Nov 02 04:00:03.025403 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.025371    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" event={"ID":"7116adfc-c3b3-47f5-949d-148d52c37b5a","Type":"ContainerStarted","Data":"a72d5ed51f72d92b71a0284c05f41126dc19f6f9eb3fcb920da6075a9aa88ec2"}
Nov 02 04:00:03.026973 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.026947    4407 generic.go:356] "Generic (PLEG): container finished" podID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerID="d25af384404e6732873e3a270e794e08e1587dcc140f7fc1b77c682089428941" exitCode=60
Nov 02 04:00:03.027097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.027030    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" event={"ID":"17485476-25e7-44cf-81d1-a485ae79f1ec","Type":"ContainerDied","Data":"d25af384404e6732873e3a270e794e08e1587dcc140f7fc1b77c682089428941"}
Nov 02 04:00:03.027256 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.027238    4407 scope.go:122] "RemoveContainer" containerID="d25af384404e6732873e3a270e794e08e1587dcc140f7fc1b77c682089428941"
Nov 02 04:00:03.028596 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.028572    4407 generic.go:356] "Generic (PLEG): container finished" podID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerID="ee927dd8a07fbc0f277f0b46e81a07f64c4d8ee03bb4d0ac4a62f7ac5c7cde0e" exitCode=60
Nov 02 04:00:03.028730 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.028644    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" event={"ID":"332b6b2c-fd0a-4f2b-96b0-0f6e23531182","Type":"ContainerDied","Data":"ee927dd8a07fbc0f277f0b46e81a07f64c4d8ee03bb4d0ac4a62f7ac5c7cde0e"}
Nov 02 04:00:03.028919 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.028903    4407 scope.go:122] "RemoveContainer" containerID="ee927dd8a07fbc0f277f0b46e81a07f64c4d8ee03bb4d0ac4a62f7ac5c7cde0e"
Nov 02 04:00:03.030386 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.030365    4407 generic.go:356] "Generic (PLEG): container finished" podID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerID="18c5c7b3ea84fceddf41d2705fd52564a7bb74b3192322f20ef6e754856aa22b" exitCode=60
Nov 02 04:00:03.030489 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.030454    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" event={"ID":"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac","Type":"ContainerDied","Data":"18c5c7b3ea84fceddf41d2705fd52564a7bb74b3192322f20ef6e754856aa22b"}
Nov 02 04:00:03.030741 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.030651    4407 scope.go:122] "RemoveContainer" containerID="18c5c7b3ea84fceddf41d2705fd52564a7bb74b3192322f20ef6e754856aa22b"
Nov 02 04:00:03.032161 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.032142    4407 generic.go:356] "Generic (PLEG): container finished" podID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerID="801c17dc773a23f9afd39fbe734fa0da3ce3746b1fe6ba622ef765e7fe2eb3f0" exitCode=60
Nov 02 04:00:03.032261 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.032243    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" event={"ID":"cc6bbf98-5906-4d5d-8a00-70390c2b2c44","Type":"ContainerDied","Data":"801c17dc773a23f9afd39fbe734fa0da3ce3746b1fe6ba622ef765e7fe2eb3f0"}
Nov 02 04:00:03.032391 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.032379    4407 scope.go:122] "RemoveContainer" containerID="801c17dc773a23f9afd39fbe734fa0da3ce3746b1fe6ba622ef765e7fe2eb3f0"
Nov 02 04:00:03.033861 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.033842    4407 generic.go:356] "Generic (PLEG): container finished" podID="d09e8458-9745-49e1-8011-187ca69a56f1" containerID="4311d8657e2e5e0641a512d007efe37b6efc026d7df8e675fe411765e6df5ddc" exitCode=60
Nov 02 04:00:03.033942 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.033923    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" event={"ID":"d09e8458-9745-49e1-8011-187ca69a56f1","Type":"ContainerDied","Data":"4311d8657e2e5e0641a512d007efe37b6efc026d7df8e675fe411765e6df5ddc"}
Nov 02 04:00:03.034138 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.034119    4407 scope.go:122] "RemoveContainer" containerID="4311d8657e2e5e0641a512d007efe37b6efc026d7df8e675fe411765e6df5ddc"
Nov 02 04:00:03.035417 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.035400    4407 generic.go:356] "Generic (PLEG): container finished" podID="0cb04331-7309-471d-af6f-617e00bac3ce" containerID="24abfb7b2172771ef9175dc7f724e65ff19ce869950273a5e4a43a8cdcf2aae5" exitCode=60
Nov 02 04:00:03.035490 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.035422    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" event={"ID":"0cb04331-7309-471d-af6f-617e00bac3ce","Type":"ContainerDied","Data":"24abfb7b2172771ef9175dc7f724e65ff19ce869950273a5e4a43a8cdcf2aae5"}
Nov 02 04:00:03.035649 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.035617    4407 scope.go:122] "RemoveContainer" containerID="24abfb7b2172771ef9175dc7f724e65ff19ce869950273a5e4a43a8cdcf2aae5"
Nov 02 04:00:03.143232 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:03.143184    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" podStartSLOduration=3.143167825 podStartE2EDuration="3.143167825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:00:03.141656319 +0000 UTC m=+4346.937004878" watchObservedRunningTime="2026-11-02 04:00:03.143167825 +0000 UTC m=+4346.938516391"
Nov 02 04:00:04.044800 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.044766    4407 generic.go:356] "Generic (PLEG): container finished" podID="d09e8458-9745-49e1-8011-187ca69a56f1" containerID="b9ec51642408e738a2faeae9d7971164fe144c5daa5f5142166ab35e42abdd68" exitCode=60
Nov 02 04:00:04.045276 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.044833    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" event={"ID":"d09e8458-9745-49e1-8011-187ca69a56f1","Type":"ContainerDied","Data":"b9ec51642408e738a2faeae9d7971164fe144c5daa5f5142166ab35e42abdd68"}
Nov 02 04:00:04.045276 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.044904    4407 scope.go:122] "RemoveContainer" containerID="4311d8657e2e5e0641a512d007efe37b6efc026d7df8e675fe411765e6df5ddc"
Nov 02 04:00:04.045276 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.045183    4407 scope.go:122] "RemoveContainer" containerID="b9ec51642408e738a2faeae9d7971164fe144c5daa5f5142166ab35e42abdd68"
Nov 02 04:00:04.045565 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:04.045540    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn_odh-ai-gateway-infra(d09e8458-9745-49e1-8011-187ca69a56f1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" podUID="d09e8458-9745-49e1-8011-187ca69a56f1"
Nov 02 04:00:04.047066 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.047029    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" event={"ID":"0cb04331-7309-471d-af6f-617e00bac3ce","Type":"ContainerStarted","Data":"7970a16788cd253318bb38ccdc141641d22f473bf09c40205b991e93cefaaa3b"}
Nov 02 04:00:04.048829 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.048809    4407 generic.go:356] "Generic (PLEG): container finished" podID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerID="a72d5ed51f72d92b71a0284c05f41126dc19f6f9eb3fcb920da6075a9aa88ec2" exitCode=60
Nov 02 04:00:04.048928 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.048888    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" event={"ID":"7116adfc-c3b3-47f5-949d-148d52c37b5a","Type":"ContainerDied","Data":"a72d5ed51f72d92b71a0284c05f41126dc19f6f9eb3fcb920da6075a9aa88ec2"}
Nov 02 04:00:04.049097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.049080    4407 scope.go:122] "RemoveContainer" containerID="a72d5ed51f72d92b71a0284c05f41126dc19f6f9eb3fcb920da6075a9aa88ec2"
Nov 02 04:00:04.049359 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:04.049338    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v_odh-ai-gateway-infra(7116adfc-c3b3-47f5-949d-148d52c37b5a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a"
Nov 02 04:00:04.050870 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.050845    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" event={"ID":"17485476-25e7-44cf-81d1-a485ae79f1ec","Type":"ContainerStarted","Data":"b7ab2df236fafbdbe497751bfc37f9b0c7d71ff4be402608664282245a2b01c5"}
Nov 02 04:00:04.052850 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.052828    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" event={"ID":"332b6b2c-fd0a-4f2b-96b0-0f6e23531182","Type":"ContainerStarted","Data":"b5d6991017b72be0c6bc3cd10ea501e42e10b5e17800a775da5783f40f656bf6"}
Nov 02 04:00:04.054613 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.054595    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" event={"ID":"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac","Type":"ContainerStarted","Data":"3eb117b99d887f07be8fb5aac46eabeeba6bf4ddbf98da1b7e10754c6af57718"}
Nov 02 04:00:04.056359 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.056338    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" event={"ID":"cc6bbf98-5906-4d5d-8a00-70390c2b2c44","Type":"ContainerStarted","Data":"916feec6ab8a2a193b5c0cf0f425edcce0538cd0fbbefdf2ac66e4069ad931e7"}
Nov 02 04:00:04.076223 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:04.076202    4407 scope.go:122] "RemoveContainer" containerID="e8932c1666803eca43c88b841bd4770f200287dd82640001feddf5fa29525a50"
Nov 02 04:00:05.066077 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.066041    4407 generic.go:356] "Generic (PLEG): container finished" podID="0cb04331-7309-471d-af6f-617e00bac3ce" containerID="7970a16788cd253318bb38ccdc141641d22f473bf09c40205b991e93cefaaa3b" exitCode=60
Nov 02 04:00:05.066615 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.066116    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" event={"ID":"0cb04331-7309-471d-af6f-617e00bac3ce","Type":"ContainerDied","Data":"7970a16788cd253318bb38ccdc141641d22f473bf09c40205b991e93cefaaa3b"}
Nov 02 04:00:05.066615 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.066180    4407 scope.go:122] "RemoveContainer" containerID="24abfb7b2172771ef9175dc7f724e65ff19ce869950273a5e4a43a8cdcf2aae5"
Nov 02 04:00:05.066615 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.066518    4407 scope.go:122] "RemoveContainer" containerID="7970a16788cd253318bb38ccdc141641d22f473bf09c40205b991e93cefaaa3b"
Nov 02 04:00:05.066947 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:05.066921    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv_odh-ai-gateway-infra(0cb04331-7309-471d-af6f-617e00bac3ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" podUID="0cb04331-7309-471d-af6f-617e00bac3ce"
Nov 02 04:00:05.068526 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.068509    4407 scope.go:122] "RemoveContainer" containerID="a72d5ed51f72d92b71a0284c05f41126dc19f6f9eb3fcb920da6075a9aa88ec2"
Nov 02 04:00:05.068889 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:05.068868    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v_odh-ai-gateway-infra(7116adfc-c3b3-47f5-949d-148d52c37b5a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a"
Nov 02 04:00:05.070327 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.070295    4407 generic.go:356] "Generic (PLEG): container finished" podID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerID="b7ab2df236fafbdbe497751bfc37f9b0c7d71ff4be402608664282245a2b01c5" exitCode=60
Nov 02 04:00:05.070423 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.070405    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" event={"ID":"17485476-25e7-44cf-81d1-a485ae79f1ec","Type":"ContainerDied","Data":"b7ab2df236fafbdbe497751bfc37f9b0c7d71ff4be402608664282245a2b01c5"}
Nov 02 04:00:05.071009 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.070991    4407 scope.go:122] "RemoveContainer" containerID="b7ab2df236fafbdbe497751bfc37f9b0c7d71ff4be402608664282245a2b01c5"
Nov 02 04:00:05.071285 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:05.071265    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l_odh-ai-gateway-infra(17485476-25e7-44cf-81d1-a485ae79f1ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec"
Nov 02 04:00:05.072377 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.072358    4407 generic.go:356] "Generic (PLEG): container finished" podID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerID="b5d6991017b72be0c6bc3cd10ea501e42e10b5e17800a775da5783f40f656bf6" exitCode=60
Nov 02 04:00:05.072463 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.072438    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" event={"ID":"332b6b2c-fd0a-4f2b-96b0-0f6e23531182","Type":"ContainerDied","Data":"b5d6991017b72be0c6bc3cd10ea501e42e10b5e17800a775da5783f40f656bf6"}
Nov 02 04:00:05.072671 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.072657    4407 scope.go:122] "RemoveContainer" containerID="b5d6991017b72be0c6bc3cd10ea501e42e10b5e17800a775da5783f40f656bf6"
Nov 02 04:00:05.072893 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:05.072877    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h_odh-ai-gateway-infra(332b6b2c-fd0a-4f2b-96b0-0f6e23531182)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182"
Nov 02 04:00:05.074445 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.074423    4407 generic.go:356] "Generic (PLEG): container finished" podID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerID="3eb117b99d887f07be8fb5aac46eabeeba6bf4ddbf98da1b7e10754c6af57718" exitCode=60
Nov 02 04:00:05.074569 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.074553    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" event={"ID":"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac","Type":"ContainerDied","Data":"3eb117b99d887f07be8fb5aac46eabeeba6bf4ddbf98da1b7e10754c6af57718"}
Nov 02 04:00:05.074801 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.074787    4407 scope.go:122] "RemoveContainer" containerID="3eb117b99d887f07be8fb5aac46eabeeba6bf4ddbf98da1b7e10754c6af57718"
Nov 02 04:00:05.075024 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:05.075000    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2_odh-ai-gateway-infra(bd6c1b51-5a41-4c74-b01d-5bdc57d662ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac"
Nov 02 04:00:05.076504 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.076484    4407 generic.go:356] "Generic (PLEG): container finished" podID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerID="916feec6ab8a2a193b5c0cf0f425edcce0538cd0fbbefdf2ac66e4069ad931e7" exitCode=60
Nov 02 04:00:05.076603 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.076569    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" event={"ID":"cc6bbf98-5906-4d5d-8a00-70390c2b2c44","Type":"ContainerDied","Data":"916feec6ab8a2a193b5c0cf0f425edcce0538cd0fbbefdf2ac66e4069ad931e7"}
Nov 02 04:00:05.076748 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.076735    4407 scope.go:122] "RemoveContainer" containerID="916feec6ab8a2a193b5c0cf0f425edcce0538cd0fbbefdf2ac66e4069ad931e7"
Nov 02 04:00:05.076967 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:05.076945    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45_odh-ai-gateway-infra(cc6bbf98-5906-4d5d-8a00-70390c2b2c44)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44"
Nov 02 04:00:05.078409 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.078391    4407 scope.go:122] "RemoveContainer" containerID="b9ec51642408e738a2faeae9d7971164fe144c5daa5f5142166ab35e42abdd68"
Nov 02 04:00:05.078618 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:05.078599    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn_odh-ai-gateway-infra(d09e8458-9745-49e1-8011-187ca69a56f1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" podUID="d09e8458-9745-49e1-8011-187ca69a56f1"
Nov 02 04:00:05.080022 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.080006    4407 scope.go:122] "RemoveContainer" containerID="d25af384404e6732873e3a270e794e08e1587dcc140f7fc1b77c682089428941"
Nov 02 04:00:05.093564 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.093543    4407 scope.go:122] "RemoveContainer" containerID="ee927dd8a07fbc0f277f0b46e81a07f64c4d8ee03bb4d0ac4a62f7ac5c7cde0e"
Nov 02 04:00:05.109524 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.109495    4407 scope.go:122] "RemoveContainer" containerID="18c5c7b3ea84fceddf41d2705fd52564a7bb74b3192322f20ef6e754856aa22b"
Nov 02 04:00:05.124382 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:05.124359    4407 scope.go:122] "RemoveContainer" containerID="801c17dc773a23f9afd39fbe734fa0da3ce3746b1fe6ba622ef765e7fe2eb3f0"
Nov 02 04:00:05.190054 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:05.190017    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:00:06.087926 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:06.087897    4407 scope.go:122] "RemoveContainer" containerID="7970a16788cd253318bb38ccdc141641d22f473bf09c40205b991e93cefaaa3b"
Nov 02 04:00:06.088408 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:06.088284    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv_odh-ai-gateway-infra(0cb04331-7309-471d-af6f-617e00bac3ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" podUID="0cb04331-7309-471d-af6f-617e00bac3ce"
Nov 02 04:00:06.089673 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:06.089651    4407 scope.go:122] "RemoveContainer" containerID="b7ab2df236fafbdbe497751bfc37f9b0c7d71ff4be402608664282245a2b01c5"
Nov 02 04:00:06.089966 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:06.089947    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l_odh-ai-gateway-infra(17485476-25e7-44cf-81d1-a485ae79f1ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec"
Nov 02 04:00:06.091287 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:06.091266    4407 scope.go:122] "RemoveContainer" containerID="b5d6991017b72be0c6bc3cd10ea501e42e10b5e17800a775da5783f40f656bf6"
Nov 02 04:00:06.091558 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:06.091538    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h_odh-ai-gateway-infra(332b6b2c-fd0a-4f2b-96b0-0f6e23531182)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182"
Nov 02 04:00:06.092983 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:06.092962    4407 scope.go:122] "RemoveContainer" containerID="3eb117b99d887f07be8fb5aac46eabeeba6bf4ddbf98da1b7e10754c6af57718"
Nov 02 04:00:06.093239 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:06.093217    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2_odh-ai-gateway-infra(bd6c1b51-5a41-4c74-b01d-5bdc57d662ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac"
Nov 02 04:00:06.094594 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:06.094579    4407 scope.go:122] "RemoveContainer" containerID="916feec6ab8a2a193b5c0cf0f425edcce0538cd0fbbefdf2ac66e4069ad931e7"
Nov 02 04:00:06.094807 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:06.094788    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45_odh-ai-gateway-infra(cc6bbf98-5906-4d5d-8a00-70390c2b2c44)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44"
Nov 02 04:00:06.476214 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:06.476134    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:11.331547 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:11.331509    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:15.344552 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:15.344522    4407 scope.go:122] "RemoveContainer" containerID="b5d6991017b72be0c6bc3cd10ea501e42e10b5e17800a775da5783f40f656bf6"
Nov 02 04:00:16.142550 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:16.142515    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:16.158740 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:16.158711    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" event={"ID":"332b6b2c-fd0a-4f2b-96b0-0f6e23531182","Type":"ContainerStarted","Data":"4fa6f0b8c29f653f7a1ee3faf20df8cd0f2ce28ce582d1ce048e7335d7be3fee"}
Nov 02 04:00:16.349234 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:16.349191    4407 scope.go:122] "RemoveContainer" containerID="916feec6ab8a2a193b5c0cf0f425edcce0538cd0fbbefdf2ac66e4069ad931e7"
Nov 02 04:00:16.349875 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:16.349393    4407 scope.go:122] "RemoveContainer" containerID="b7ab2df236fafbdbe497751bfc37f9b0c7d71ff4be402608664282245a2b01c5"
Nov 02 04:00:16.349875 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:16.349861    4407 scope.go:122] "RemoveContainer" containerID="7970a16788cd253318bb38ccdc141641d22f473bf09c40205b991e93cefaaa3b"
Nov 02 04:00:16.351067 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:16.350471    4407 scope.go:122] "RemoveContainer" containerID="a72d5ed51f72d92b71a0284c05f41126dc19f6f9eb3fcb920da6075a9aa88ec2"
Nov 02 04:00:16.434711 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:16.434677    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:16.600776 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:16.600749    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:17.168215 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.168175    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" event={"ID":"0cb04331-7309-471d-af6f-617e00bac3ce","Type":"ContainerStarted","Data":"6eb8a6919758dadf7a9507689f50a45ce418aab8d36d900944b4b55a262697dc"}
Nov 02 04:00:17.170351 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.170323    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" event={"ID":"7116adfc-c3b3-47f5-949d-148d52c37b5a","Type":"ContainerStarted","Data":"fb0e20e6e8ae3fb03a9c09e99d00754ef74dd2bc9463490c40eaa507e6c510a1"}
Nov 02 04:00:17.172754 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.172725    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" event={"ID":"17485476-25e7-44cf-81d1-a485ae79f1ec","Type":"ContainerStarted","Data":"e526aec086b3f8cf1d543b55bef84457af730f8523b8a2c2c8c022b122d515f4"}
Nov 02 04:00:17.174733 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.174704    4407 generic.go:356] "Generic (PLEG): container finished" podID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerID="4fa6f0b8c29f653f7a1ee3faf20df8cd0f2ce28ce582d1ce048e7335d7be3fee" exitCode=60
Nov 02 04:00:17.174830 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.174769    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" event={"ID":"332b6b2c-fd0a-4f2b-96b0-0f6e23531182","Type":"ContainerDied","Data":"4fa6f0b8c29f653f7a1ee3faf20df8cd0f2ce28ce582d1ce048e7335d7be3fee"}
Nov 02 04:00:17.174830 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.174813    4407 scope.go:122] "RemoveContainer" containerID="b5d6991017b72be0c6bc3cd10ea501e42e10b5e17800a775da5783f40f656bf6"
Nov 02 04:00:17.175128 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.175112    4407 scope.go:122] "RemoveContainer" containerID="4fa6f0b8c29f653f7a1ee3faf20df8cd0f2ce28ce582d1ce048e7335d7be3fee"
Nov 02 04:00:17.175407 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:17.175370    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h_odh-ai-gateway-infra(332b6b2c-fd0a-4f2b-96b0-0f6e23531182)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182"
Nov 02 04:00:17.177516 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.177489    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" event={"ID":"cc6bbf98-5906-4d5d-8a00-70390c2b2c44","Type":"ContainerStarted","Data":"08a71391d3c84b983cb30791a58f55155d87d80d35d5c3c61fc4c448b60ad624"}
Nov 02 04:00:17.292580 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.292547    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"]
Nov 02 04:00:17.344363 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.344334    4407 scope.go:122] "RemoveContainer" containerID="3eb117b99d887f07be8fb5aac46eabeeba6bf4ddbf98da1b7e10754c6af57718"
Nov 02 04:00:17.344543 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:17.344447    4407 scope.go:122] "RemoveContainer" containerID="b9ec51642408e738a2faeae9d7971164fe144c5daa5f5142166ab35e42abdd68"
Nov 02 04:00:18.187050 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.186945    4407 generic.go:356] "Generic (PLEG): container finished" podID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerID="08a71391d3c84b983cb30791a58f55155d87d80d35d5c3c61fc4c448b60ad624" exitCode=60
Nov 02 04:00:18.187050 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.187028    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" event={"ID":"cc6bbf98-5906-4d5d-8a00-70390c2b2c44","Type":"ContainerDied","Data":"08a71391d3c84b983cb30791a58f55155d87d80d35d5c3c61fc4c448b60ad624"}
Nov 02 04:00:18.187546 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.187102    4407 scope.go:122] "RemoveContainer" containerID="916feec6ab8a2a193b5c0cf0f425edcce0538cd0fbbefdf2ac66e4069ad931e7"
Nov 02 04:00:18.187546 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.187479    4407 scope.go:122] "RemoveContainer" containerID="08a71391d3c84b983cb30791a58f55155d87d80d35d5c3c61fc4c448b60ad624"
Nov 02 04:00:18.187847 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:18.187822    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45_odh-ai-gateway-infra(cc6bbf98-5906-4d5d-8a00-70390c2b2c44)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44"
Nov 02 04:00:18.189161 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.189138    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" event={"ID":"d09e8458-9745-49e1-8011-187ca69a56f1","Type":"ContainerStarted","Data":"9f3a96a0bebfa6139e7b5f593c08a402ff41d47e4e1a79fa1316c4b1c9ebc77c"}
Nov 02 04:00:18.190985 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.190965    4407 generic.go:356] "Generic (PLEG): container finished" podID="0cb04331-7309-471d-af6f-617e00bac3ce" containerID="6eb8a6919758dadf7a9507689f50a45ce418aab8d36d900944b4b55a262697dc" exitCode=60
Nov 02 04:00:18.191086 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.191044    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" event={"ID":"0cb04331-7309-471d-af6f-617e00bac3ce","Type":"ContainerDied","Data":"6eb8a6919758dadf7a9507689f50a45ce418aab8d36d900944b4b55a262697dc"}
Nov 02 04:00:18.191543 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.191525    4407 scope.go:122] "RemoveContainer" containerID="6eb8a6919758dadf7a9507689f50a45ce418aab8d36d900944b4b55a262697dc"
Nov 02 04:00:18.191829 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:18.191801    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv_odh-ai-gateway-infra(0cb04331-7309-471d-af6f-617e00bac3ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" podUID="0cb04331-7309-471d-af6f-617e00bac3ce"
Nov 02 04:00:18.193068 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.193050    4407 generic.go:356] "Generic (PLEG): container finished" podID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerID="fb0e20e6e8ae3fb03a9c09e99d00754ef74dd2bc9463490c40eaa507e6c510a1" exitCode=60
Nov 02 04:00:18.193168 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.193107    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" event={"ID":"7116adfc-c3b3-47f5-949d-148d52c37b5a","Type":"ContainerDied","Data":"fb0e20e6e8ae3fb03a9c09e99d00754ef74dd2bc9463490c40eaa507e6c510a1"}
Nov 02 04:00:18.193444 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.193424    4407 scope.go:122] "RemoveContainer" containerID="fb0e20e6e8ae3fb03a9c09e99d00754ef74dd2bc9463490c40eaa507e6c510a1"
Nov 02 04:00:18.193671 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:18.193648    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v_odh-ai-gateway-infra(7116adfc-c3b3-47f5-949d-148d52c37b5a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a"
Nov 02 04:00:18.195090 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.195062    4407 generic.go:356] "Generic (PLEG): container finished" podID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerID="e526aec086b3f8cf1d543b55bef84457af730f8523b8a2c2c8c022b122d515f4" exitCode=60
Nov 02 04:00:18.195186 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.195126    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" event={"ID":"17485476-25e7-44cf-81d1-a485ae79f1ec","Type":"ContainerDied","Data":"e526aec086b3f8cf1d543b55bef84457af730f8523b8a2c2c8c022b122d515f4"}
Nov 02 04:00:18.195453 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.195433    4407 scope.go:122] "RemoveContainer" containerID="e526aec086b3f8cf1d543b55bef84457af730f8523b8a2c2c8c022b122d515f4"
Nov 02 04:00:18.195713 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:18.195692    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l_odh-ai-gateway-infra(17485476-25e7-44cf-81d1-a485ae79f1ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec"
Nov 02 04:00:18.198874 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.198853    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" event={"ID":"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac","Type":"ContainerStarted","Data":"b4dda0dfeb903bf3772219ccedbd0479c13c05b19947af0849851fd2cd99fbe0"}
Nov 02 04:00:18.201175 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.201152    4407 scope.go:122] "RemoveContainer" containerID="7970a16788cd253318bb38ccdc141641d22f473bf09c40205b991e93cefaaa3b"
Nov 02 04:00:18.214712 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.214682    4407 scope.go:122] "RemoveContainer" containerID="a72d5ed51f72d92b71a0284c05f41126dc19f6f9eb3fcb920da6075a9aa88ec2"
Nov 02 04:00:18.228957 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.228935    4407 scope.go:122] "RemoveContainer" containerID="b7ab2df236fafbdbe497751bfc37f9b0c7d71ff4be402608664282245a2b01c5"
Nov 02 04:00:18.313458 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.313430    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"]
Nov 02 04:00:18.356728 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.356706    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:18.429996 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.429971    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"]
Nov 02 04:00:18.481118 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.481046    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-maas-api-service-ca\") pod \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\" (UID: \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\") "
Nov 02 04:00:18.481286 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.481164    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-kube-api-access-8tk42\" (UniqueName: \"kubernetes.io/projected/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-kube-api-access-8tk42\") pod \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\" (UID: \"332b6b2c-fd0a-4f2b-96b0-0f6e23531182\") "
Nov 02 04:00:18.481444 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.481422    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-maas-api-service-ca" pod "332b6b2c-fd0a-4f2b-96b0-0f6e23531182" (UID: "332b6b2c-fd0a-4f2b-96b0-0f6e23531182"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:00:18.481897 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.481873    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-maas-api-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:18.483303 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.483281    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-kube-api-access-8tk42" pod "332b6b2c-fd0a-4f2b-96b0-0f6e23531182" (UID: "332b6b2c-fd0a-4f2b-96b0-0f6e23531182"). InnerVolumeSpecName "kube-api-access-8tk42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:00:18.543115 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.543081    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"]
Nov 02 04:00:18.583197 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.583166    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tk42\" (UniqueName: \"kubernetes.io/projected/332b6b2c-fd0a-4f2b-96b0-0f6e23531182-kube-api-access-8tk42\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:18.669464 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:18.669427    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"]
Nov 02 04:00:19.207063 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.206964    4407 generic.go:356] "Generic (PLEG): container finished" podID="d09e8458-9745-49e1-8011-187ca69a56f1" containerID="9f3a96a0bebfa6139e7b5f593c08a402ff41d47e4e1a79fa1316c4b1c9ebc77c" exitCode=60
Nov 02 04:00:19.207527 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.207081    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" event={"ID":"d09e8458-9745-49e1-8011-187ca69a56f1","Type":"ContainerDied","Data":"9f3a96a0bebfa6139e7b5f593c08a402ff41d47e4e1a79fa1316c4b1c9ebc77c"}
Nov 02 04:00:19.207527 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.207157    4407 scope.go:122] "RemoveContainer" containerID="b9ec51642408e738a2faeae9d7971164fe144c5daa5f5142166ab35e42abdd68"
Nov 02 04:00:19.207720 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.207549    4407 scope.go:122] "RemoveContainer" containerID="9f3a96a0bebfa6139e7b5f593c08a402ff41d47e4e1a79fa1316c4b1c9ebc77c"
Nov 02 04:00:19.207949 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:19.207923    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn_odh-ai-gateway-infra(d09e8458-9745-49e1-8011-187ca69a56f1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" podUID="d09e8458-9745-49e1-8011-187ca69a56f1"
Nov 02 04:00:19.214745 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.214725    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"
Nov 02 04:00:19.215040 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.214749    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h" event={"ID":"332b6b2c-fd0a-4f2b-96b0-0f6e23531182","Type":"ContainerDied","Data":"a767d3f1114d0a1f8db2ff9acf5b40644e8f9c72a1c63b10091619ee08947311"}
Nov 02 04:00:19.217604 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.217582    4407 generic.go:356] "Generic (PLEG): container finished" podID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerID="b4dda0dfeb903bf3772219ccedbd0479c13c05b19947af0849851fd2cd99fbe0" exitCode=60
Nov 02 04:00:19.217732 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.217672    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" event={"ID":"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac","Type":"ContainerDied","Data":"b4dda0dfeb903bf3772219ccedbd0479c13c05b19947af0849851fd2cd99fbe0"}
Nov 02 04:00:19.218090 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.218069    4407 scope.go:122] "RemoveContainer" containerID="b4dda0dfeb903bf3772219ccedbd0479c13c05b19947af0849851fd2cd99fbe0"
Nov 02 04:00:19.218422 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:19.218393    4407 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2_odh-ai-gateway-infra(bd6c1b51-5a41-4c74-b01d-5bdc57d662ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac"
Nov 02 04:00:19.224801 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.224784    4407 scope.go:122] "RemoveContainer" containerID="4fa6f0b8c29f653f7a1ee3faf20df8cd0f2ce28ce582d1ce048e7335d7be3fee"
Nov 02 04:00:19.238839 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.238811    4407 scope.go:122] "RemoveContainer" containerID="3eb117b99d887f07be8fb5aac46eabeeba6bf4ddbf98da1b7e10754c6af57718"
Nov 02 04:00:19.399370 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.399346    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:19.435329 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.435301    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:19.451157 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.451129    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"]
Nov 02 04:00:19.456071 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.456045    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-7c1bee-29893200-5qk2h"]
Nov 02 04:00:19.496787 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.496767    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7116adfc-c3b3-47f5-949d-148d52c37b5a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7116adfc-c3b3-47f5-949d-148d52c37b5a-maas-api-service-ca\") pod \"7116adfc-c3b3-47f5-949d-148d52c37b5a\" (UID: \"7116adfc-c3b3-47f5-949d-148d52c37b5a\") "
Nov 02 04:00:19.496973 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.496958    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7116adfc-c3b3-47f5-949d-148d52c37b5a-kube-api-access-j8xn8\" (UniqueName: \"kubernetes.io/projected/7116adfc-c3b3-47f5-949d-148d52c37b5a-kube-api-access-j8xn8\") pod \"7116adfc-c3b3-47f5-949d-148d52c37b5a\" (UID: \"7116adfc-c3b3-47f5-949d-148d52c37b5a\") "
Nov 02 04:00:19.497202 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.497175    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7116adfc-c3b3-47f5-949d-148d52c37b5a-maas-api-service-ca" pod "7116adfc-c3b3-47f5-949d-148d52c37b5a" (UID: "7116adfc-c3b3-47f5-949d-148d52c37b5a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:00:19.497567 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.497549    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7116adfc-c3b3-47f5-949d-148d52c37b5a-maas-api-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:19.499238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.499218    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7116adfc-c3b3-47f5-949d-148d52c37b5a-kube-api-access-j8xn8" pod "7116adfc-c3b3-47f5-949d-148d52c37b5a" (UID: "7116adfc-c3b3-47f5-949d-148d52c37b5a"). InnerVolumeSpecName "kube-api-access-j8xn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:00:19.520090 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.520069    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:19.523061 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.523045    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:19.544971 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.544944    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"]
Nov 02 04:00:19.598143 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.598117    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cb04331-7309-471d-af6f-617e00bac3ce-kube-api-access-hlcwr\" (UniqueName: \"kubernetes.io/projected/0cb04331-7309-471d-af6f-617e00bac3ce-kube-api-access-hlcwr\") pod \"0cb04331-7309-471d-af6f-617e00bac3ce\" (UID: \"0cb04331-7309-471d-af6f-617e00bac3ce\") "
Nov 02 04:00:19.598281 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.598231    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0cb04331-7309-471d-af6f-617e00bac3ce-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb04331-7309-471d-af6f-617e00bac3ce-maas-api-service-ca\") pod \"0cb04331-7309-471d-af6f-617e00bac3ce\" (UID: \"0cb04331-7309-471d-af6f-617e00bac3ce\") "
Nov 02 04:00:19.598582 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.598558    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb04331-7309-471d-af6f-617e00bac3ce-maas-api-service-ca" pod "0cb04331-7309-471d-af6f-617e00bac3ce" (UID: "0cb04331-7309-471d-af6f-617e00bac3ce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:00:19.598582 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.598576    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8xn8\" (UniqueName: \"kubernetes.io/projected/7116adfc-c3b3-47f5-949d-148d52c37b5a-kube-api-access-j8xn8\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:19.600318 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.600289    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb04331-7309-471d-af6f-617e00bac3ce-kube-api-access-hlcwr" pod "0cb04331-7309-471d-af6f-617e00bac3ce" (UID: "0cb04331-7309-471d-af6f-617e00bac3ce"). InnerVolumeSpecName "kube-api-access-hlcwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:00:19.699204 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.699177    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-maas-api-service-ca\") pod \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\" (UID: \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\") "
Nov 02 04:00:19.699377 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.699290    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17485476-25e7-44cf-81d1-a485ae79f1ec-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17485476-25e7-44cf-81d1-a485ae79f1ec-maas-api-service-ca\") pod \"17485476-25e7-44cf-81d1-a485ae79f1ec\" (UID: \"17485476-25e7-44cf-81d1-a485ae79f1ec\") "
Nov 02 04:00:19.699377 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.699375    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-kube-api-access-rvm29\" (UniqueName: \"kubernetes.io/projected/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-kube-api-access-rvm29\") pod \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\" (UID: \"cc6bbf98-5906-4d5d-8a00-70390c2b2c44\") "
Nov 02 04:00:19.699456 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.699446    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17485476-25e7-44cf-81d1-a485ae79f1ec-kube-api-access-5pkpb\" (UniqueName: \"kubernetes.io/projected/17485476-25e7-44cf-81d1-a485ae79f1ec-kube-api-access-5pkpb\") pod \"17485476-25e7-44cf-81d1-a485ae79f1ec\" (UID: \"17485476-25e7-44cf-81d1-a485ae79f1ec\") "
Nov 02 04:00:19.699672 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.699601    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-maas-api-service-ca" pod "cc6bbf98-5906-4d5d-8a00-70390c2b2c44" (UID: "cc6bbf98-5906-4d5d-8a00-70390c2b2c44"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:00:19.699746 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.699708    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17485476-25e7-44cf-81d1-a485ae79f1ec-maas-api-service-ca" pod "17485476-25e7-44cf-81d1-a485ae79f1ec" (UID: "17485476-25e7-44cf-81d1-a485ae79f1ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:00:19.699993 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.699979    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb04331-7309-471d-af6f-617e00bac3ce-maas-api-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:19.700030 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.700001    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-maas-api-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:19.700030 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.700011    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17485476-25e7-44cf-81d1-a485ae79f1ec-maas-api-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:19.700030 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.700020    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlcwr\" (UniqueName: \"kubernetes.io/projected/0cb04331-7309-471d-af6f-617e00bac3ce-kube-api-access-hlcwr\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:19.701615 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.701595    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17485476-25e7-44cf-81d1-a485ae79f1ec-kube-api-access-5pkpb" pod "17485476-25e7-44cf-81d1-a485ae79f1ec" (UID: "17485476-25e7-44cf-81d1-a485ae79f1ec"). InnerVolumeSpecName "kube-api-access-5pkpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:00:19.701691 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.701614    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-kube-api-access-rvm29" pod "cc6bbf98-5906-4d5d-8a00-70390c2b2c44" (UID: "cc6bbf98-5906-4d5d-8a00-70390c2b2c44"). InnerVolumeSpecName "kube-api-access-rvm29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:00:19.779526 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.779497    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"]
Nov 02 04:00:19.801126 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.801101    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvm29\" (UniqueName: \"kubernetes.io/projected/cc6bbf98-5906-4d5d-8a00-70390c2b2c44-kube-api-access-rvm29\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:19.801126 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:19.801128    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pkpb\" (UniqueName: \"kubernetes.io/projected/17485476-25e7-44cf-81d1-a485ae79f1ec-kube-api-access-5pkpb\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:20.232916 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.232821    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"
Nov 02 04:00:20.232916 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.232838    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l" event={"ID":"17485476-25e7-44cf-81d1-a485ae79f1ec","Type":"ContainerDied","Data":"3b41a6157da79d80902c77ab13da3852d26f5bfe537238d0033aff2afe9df56a"}
Nov 02 04:00:20.233433 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.232947    4407 scope.go:122] "RemoveContainer" containerID="e526aec086b3f8cf1d543b55bef84457af730f8523b8a2c2c8c022b122d515f4"
Nov 02 04:00:20.237413 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.237392    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"
Nov 02 04:00:20.237576 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.237552    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45" event={"ID":"cc6bbf98-5906-4d5d-8a00-70390c2b2c44","Type":"ContainerDied","Data":"8c84f8f768cd723edfdd4e0d423eda566332b244a640ffbd99e86ca156aabeb1"}
Nov 02 04:00:20.246133 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.246116    4407 scope.go:122] "RemoveContainer" containerID="08a71391d3c84b983cb30791a58f55155d87d80d35d5c3c61fc4c448b60ad624"
Nov 02 04:00:20.246863 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.246844    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"
Nov 02 04:00:20.246964 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.246842    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv" event={"ID":"0cb04331-7309-471d-af6f-617e00bac3ce","Type":"ContainerDied","Data":"ae1c2e9bfec09146025e86fd97ffff89e34a7b2df2c7c1b2d0ce84b6d42330c6"}
Nov 02 04:00:20.248708 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.248679    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v" event={"ID":"7116adfc-c3b3-47f5-949d-148d52c37b5a","Type":"ContainerDied","Data":"f2550cdec51d15de05143674ae2a48efc1082dd524de61f195042be7bfc6ba07"}
Nov 02 04:00:20.248807 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.248791    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"
Nov 02 04:00:20.264941 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.262968    4407 scope.go:122] "RemoveContainer" containerID="6eb8a6919758dadf7a9507689f50a45ce418aab8d36d900944b4b55a262697dc"
Nov 02 04:00:20.277097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.275398    4407 scope.go:122] "RemoveContainer" containerID="fb0e20e6e8ae3fb03a9c09e99d00754ef74dd2bc9463490c40eaa507e6c510a1"
Nov 02 04:00:20.357089 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.357066    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" path="/var/lib/kubelet/pods/332b6b2c-fd0a-4f2b-96b0-0f6e23531182/volumes"
Nov 02 04:00:20.428689 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.428653    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:20.469285 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.469264    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:20.612493 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.612465    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-maas-api-service-ca\") pod \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\" (UID: \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\") "
Nov 02 04:00:20.612742 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.612665    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d09e8458-9745-49e1-8011-187ca69a56f1-kube-api-access-8h8n9\" (UniqueName: \"kubernetes.io/projected/d09e8458-9745-49e1-8011-187ca69a56f1-kube-api-access-8h8n9\") pod \"d09e8458-9745-49e1-8011-187ca69a56f1\" (UID: \"d09e8458-9745-49e1-8011-187ca69a56f1\") "
Nov 02 04:00:20.612742 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.612731    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d09e8458-9745-49e1-8011-187ca69a56f1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09e8458-9745-49e1-8011-187ca69a56f1-maas-api-service-ca\") pod \"d09e8458-9745-49e1-8011-187ca69a56f1\" (UID: \"d09e8458-9745-49e1-8011-187ca69a56f1\") "
Nov 02 04:00:20.612877 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.612788    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-kube-api-access-9pdb6\" (UniqueName: \"kubernetes.io/projected/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-kube-api-access-9pdb6\") pod \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\" (UID: \"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac\") "
Nov 02 04:00:20.612934 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.612887    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-maas-api-service-ca" pod "bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" (UID: "bd6c1b51-5a41-4c74-b01d-5bdc57d662ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:00:20.613194 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.613171    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d09e8458-9745-49e1-8011-187ca69a56f1-maas-api-service-ca" pod "d09e8458-9745-49e1-8011-187ca69a56f1" (UID: "d09e8458-9745-49e1-8011-187ca69a56f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:00:20.613282 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.613265    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-maas-api-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:20.613323 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.613291    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09e8458-9745-49e1-8011-187ca69a56f1-maas-api-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:20.615193 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.615173    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d09e8458-9745-49e1-8011-187ca69a56f1-kube-api-access-8h8n9" pod "d09e8458-9745-49e1-8011-187ca69a56f1" (UID: "d09e8458-9745-49e1-8011-187ca69a56f1"). InnerVolumeSpecName "kube-api-access-8h8n9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:00:20.615477 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.615461    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-kube-api-access-9pdb6" pod "bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" (UID: "bd6c1b51-5a41-4c74-b01d-5bdc57d662ac"). InnerVolumeSpecName "kube-api-access-9pdb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:00:20.636103 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.636078    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"]
Nov 02 04:00:20.641908 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.641880    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-5ac93f-29893200-kmj45"]
Nov 02 04:00:20.715040 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.715010    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h8n9\" (UniqueName: \"kubernetes.io/projected/d09e8458-9745-49e1-8011-187ca69a56f1-kube-api-access-8h8n9\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:20.715040 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.715035    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pdb6\" (UniqueName: \"kubernetes.io/projected/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac-kube-api-access-9pdb6\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:20.867960 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.867879    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"]
Nov 02 04:00:20.872256 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:20.872230    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-a7cf93-29893200-2nd9v"]
Nov 02 04:00:21.095647 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.095594    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"]
Nov 02 04:00:21.098487 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.098466    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-3d14ee-29893200-q7khv"]
Nov 02 04:00:21.209151 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.209069    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"]
Nov 02 04:00:21.213876 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.213850    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-858236-29893200-xm97l"]
Nov 02 04:00:21.260139 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.260109    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"
Nov 02 04:00:21.260590 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.260109    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2" event={"ID":"bd6c1b51-5a41-4c74-b01d-5bdc57d662ac","Type":"ContainerDied","Data":"afe9bc7380006b54ec887046b5bbe4579a2aa2a289827bff1ed6ca811201637c"}
Nov 02 04:00:21.260590 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.260251    4407 scope.go:122] "RemoveContainer" containerID="b4dda0dfeb903bf3772219ccedbd0479c13c05b19947af0849851fd2cd99fbe0"
Nov 02 04:00:21.271048 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.270017    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn" event={"ID":"d09e8458-9745-49e1-8011-187ca69a56f1","Type":"ContainerDied","Data":"bb43c622f577e5e1c2af97e26e9c10cd703532d4a59316720919768fe8306829"}
Nov 02 04:00:21.271048 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.270127    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"
Nov 02 04:00:21.278826 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.278808    4407 scope.go:122] "RemoveContainer" containerID="9f3a96a0bebfa6139e7b5f593c08a402ff41d47e4e1a79fa1316c4b1c9ebc77c"
Nov 02 04:00:21.415683 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.415657    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:21.426794 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.426767    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"]
Nov 02 04:00:21.429687 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.429665    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-9b7f43-29893200-ptjtn"]
Nov 02 04:00:21.541844 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.541813    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"]
Nov 02 04:00:21.544548 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:21.544529    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ca6546-29893200-sxkd2"]
Nov 02 04:00:22.351922 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:22.351888    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" path="/var/lib/kubelet/pods/0cb04331-7309-471d-af6f-617e00bac3ce/volumes"
Nov 02 04:00:22.352305 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:22.352273    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" path="/var/lib/kubelet/pods/17485476-25e7-44cf-81d1-a485ae79f1ec/volumes"
Nov 02 04:00:22.352667 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:22.352653    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" path="/var/lib/kubelet/pods/7116adfc-c3b3-47f5-949d-148d52c37b5a/volumes"
Nov 02 04:00:22.353334 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:22.353302    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" path="/var/lib/kubelet/pods/bd6c1b51-5a41-4c74-b01d-5bdc57d662ac/volumes"
Nov 02 04:00:22.353716 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:22.353693    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" path="/var/lib/kubelet/pods/cc6bbf98-5906-4d5d-8a00-70390c2b2c44/volumes"
Nov 02 04:00:22.354046 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:22.354034    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" path="/var/lib/kubelet/pods/d09e8458-9745-49e1-8011-187ca69a56f1/volumes"
Nov 02 04:00:24.467421 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:24.467386    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:00:26.291466 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:26.283563    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:31.130525 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:31.130488    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:35.190668 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:35.190611    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:00:36.385751 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:36.385724    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:41.379301 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:41.379268    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:42.561712 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:42.561670    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:42.749229 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:42.749200    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:46.390391 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:46.390354    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:51.226442 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:51.226406    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:52.149108 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:52.149069    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"]
Nov 02 04:00:52.536610 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:52.536579    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:52.707517 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:52.707481    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:53.131507 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:53.131463    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:55.556691 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:55.556587    4407 generic.go:356] "Generic (PLEG): container finished" podID="dfb3623e-4997-4231-9052-54b65d6441f1" containerID="8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d" exitCode=0
Nov 02 04:00:55.556691 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:55.556667    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" event={"ID":"dfb3623e-4997-4231-9052-54b65d6441f1","Type":"ContainerDied","Data":"8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d"}
Nov 02 04:00:55.557399 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:55.557191    4407 scope.go:122] "RemoveContainer" containerID="8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d"
Nov 02 04:00:55.559311 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:55.559282    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:00:55.559383 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:55.559354    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:00:55.559430 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:55.559384    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:00:56.054022 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:56.053993    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:00:56.456269 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:56.456180    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 04:00:56.564694 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:56.564656    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" event={"ID":"dfb3623e-4997-4231-9052-54b65d6441f1","Type":"ContainerStarted","Data":"98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22"}
Nov 02 04:00:56.745330 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:56.745304    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 04:00:56.746467 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:56.746441    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 02 04:00:56.746594 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:56.746485    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 02 04:00:57.149984 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.149859    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r" podUID="766d01ef-6350-4f20-a5a6-b8e95afdae9b" containerName="payload-processing" containerID="cri-o://b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159" gracePeriod=25
Nov 02 04:00:57.442873 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.442847    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 04:00:57.513209 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.513174    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/766d01ef-6350-4f20-a5a6-b8e95afdae9b-kube-api-access-4mtkq\" (UniqueName: \"kubernetes.io/projected/766d01ef-6350-4f20-a5a6-b8e95afdae9b-kube-api-access-4mtkq\") pod \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\" (UID: \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\") "
Nov 02 04:00:57.513413 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.513299    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/766d01ef-6350-4f20-a5a6-b8e95afdae9b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d01ef-6350-4f20-a5a6-b8e95afdae9b-plugins-config-volume\") pod \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\" (UID: \"766d01ef-6350-4f20-a5a6-b8e95afdae9b\") "
Nov 02 04:00:57.513730 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.513708    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/766d01ef-6350-4f20-a5a6-b8e95afdae9b-plugins-config-volume" pod "766d01ef-6350-4f20-a5a6-b8e95afdae9b" (UID: "766d01ef-6350-4f20-a5a6-b8e95afdae9b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:00:57.515457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.515434    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/766d01ef-6350-4f20-a5a6-b8e95afdae9b-kube-api-access-4mtkq" pod "766d01ef-6350-4f20-a5a6-b8e95afdae9b" (UID: "766d01ef-6350-4f20-a5a6-b8e95afdae9b"). InnerVolumeSpecName "kube-api-access-4mtkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:00:57.572833 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.572797    4407 generic.go:356] "Generic (PLEG): container finished" podID="766d01ef-6350-4f20-a5a6-b8e95afdae9b" containerID="b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159" exitCode=0
Nov 02 04:00:57.573228 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.572856    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"
Nov 02 04:00:57.573228 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.572877    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r" event={"ID":"766d01ef-6350-4f20-a5a6-b8e95afdae9b","Type":"ContainerDied","Data":"b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159"}
Nov 02 04:00:57.573228 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.572930    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r" event={"ID":"766d01ef-6350-4f20-a5a6-b8e95afdae9b","Type":"ContainerDied","Data":"deb73e12d74b015068fc93b3f1c97fcb9fd5588d7c4cb0b14800dc6ea86b9c52"}
Nov 02 04:00:57.573228 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.572957    4407 scope.go:122] "RemoveContainer" containerID="b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159"
Nov 02 04:00:57.611876 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.611854    4407 scope.go:122] "RemoveContainer" containerID="b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159"
Nov 02 04:00:57.612224 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:00:57.612198    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159\": container with ID starting with b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159 not found: ID does not exist" containerID="b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159"
Nov 02 04:00:57.612320 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.612232    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159"} err="failed to get container status \"b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159\": rpc error: code = NotFound desc = could not find container \"b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159\": container with ID starting with b4be617e14bea8c30892418aba5873e1678b626f93628a730412a0ebb5fe8159 not found: ID does not exist"
Nov 02 04:00:57.614522 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.614448    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d01ef-6350-4f20-a5a6-b8e95afdae9b-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:57.614522 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.614478    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mtkq\" (UniqueName: \"kubernetes.io/projected/766d01ef-6350-4f20-a5a6-b8e95afdae9b-kube-api-access-4mtkq\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:00:57.742158 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.742071    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"]
Nov 02 04:00:57.745311 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.745276    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 02 04:00:57.745480 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.745325    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 02 04:00:57.748395 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:57.748376    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9b7f43-8fbfd5769-xkq9r"]
Nov 02 04:00:58.352696 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:58.352665    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="766d01ef-6350-4f20-a5a6-b8e95afdae9b" path="/var/lib/kubelet/pods/766d01ef-6350-4f20-a5a6-b8e95afdae9b/volumes"
Nov 02 04:00:58.745578 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:58.745547    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 02 04:00:58.746018 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:58.745610    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 02 04:00:59.744895 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:59.744851    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 02 04:00:59.745109 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:00:59.744912    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 02 04:01:00.569359 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569324    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"]
Nov 02 04:01:00.569926 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569911    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:01:00.569967 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569927    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:01:00.569967 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569944    4407 cpu_manager.go:414] "containerMap: removing container" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.569967 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569949    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.569967 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569959    4407 cpu_manager.go:414] "containerMap: removing container" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.569967 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569965    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569976    4407 cpu_manager.go:414] "containerMap: removing container" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569984    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.569996    4407 cpu_manager.go:414] "containerMap: removing container" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570000    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570011    4407 cpu_manager.go:414] "containerMap: removing container" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570016    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570023    4407 cpu_manager.go:414] "containerMap: removing container" podUID="766d01ef-6350-4f20-a5a6-b8e95afdae9b" containerName="payload-processing"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570027    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="766d01ef-6350-4f20-a5a6-b8e95afdae9b" containerName="payload-processing"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570032    4407 cpu_manager.go:414] "containerMap: removing container" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570037    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570043    4407 cpu_manager.go:414] "containerMap: removing container" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570047    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570052    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570056    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570066    4407 cpu_manager.go:414] "containerMap: removing container" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570070    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570077    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570081    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570088    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570091    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570097    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570101    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570106    4407 cpu_manager.go:414] "containerMap: removing container" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570110    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570122    4407 cpu_manager.go:414] "containerMap: removing container" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570126    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.570121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570131    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570136    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570147    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570152    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570157    4407 cpu_manager.go:414] "containerMap: removing container" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570161    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570168    4407 cpu_manager.go:414] "containerMap: removing container" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570172    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570250    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570256    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570263    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570270    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570275    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="766d01ef-6350-4f20-a5a6-b8e95afdae9b" containerName="payload-processing"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570280    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570285    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570290    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570294    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570299    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570304    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570309    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570314    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="332b6b2c-fd0a-4f2b-96b0-0f6e23531182" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570319    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570325    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570329    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="17485476-25e7-44cf-81d1-a485ae79f1ec" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570334    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570339    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570344    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:01:00.571039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.570350    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc6bbf98-5906-4d5d-8a00-70390c2b2c44" containerName="cleanup"
Nov 02 04:01:00.573219 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.573203    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.643537 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.643503    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-metrics-certs\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.643698 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.643653    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.643769 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.643731    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65lsp\" (UniqueName: \"kubernetes.io/projected/cfed1752-5a72-44c9-ad37-86387277f9ac-kube-api-access-65lsp\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.643916 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.643888    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfed1752-5a72-44c9-ad37-86387277f9ac-openshift-service-ca\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.705652 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.694181    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-e69cf2-serving-cert\""
Nov 02 04:01:00.712860 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.712829    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"]
Nov 02 04:01:00.744676 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.744650    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 02 04:01:00.744849 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.744702    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 02 04:01:00.744985 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.744964    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfed1752-5a72-44c9-ad37-86387277f9ac-openshift-service-ca\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.745059 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.745044    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-metrics-certs\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.745101 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.745090    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.745140 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.745130    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65lsp\" (UniqueName: \"kubernetes.io/projected/cfed1752-5a72-44c9-ad37-86387277f9ac-kube-api-access-65lsp\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.745273 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:01:00.745253    4407 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-serving-cert: secret "maas-api-e2e-autores-e69cf2-serving-cert" not found
Nov 02 04:01:00.745357 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:01:00.745336    4407 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls podName:cfed1752-5a72-44c9-ad37-86387277f9ac nodeName:}" failed. No retries permitted until 2026-11-02 04:01:01.245315694 +0000 UTC m=+4405.040664254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls") pod "maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj" (UID: "cfed1752-5a72-44c9-ad37-86387277f9ac") : secret "maas-api-e2e-autores-e69cf2-serving-cert" not found
Nov 02 04:01:00.745676 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.745658    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfed1752-5a72-44c9-ad37-86387277f9ac-openshift-service-ca\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.747516 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.747499    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-metrics-certs\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.808308 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.808272    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:00.808521 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.808408    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65lsp\" (UniqueName: \"kubernetes.io/projected/cfed1752-5a72-44c9-ad37-86387277f9ac-kube-api-access-65lsp\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:00.970907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:00.970794    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:01.144525 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.144464    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:01.251192 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.251157    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:01.253649 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.253609    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls\") pod \"maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:01.503704 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.499863    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:01.795699 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.795609    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 04:01:01.796329 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.796046    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 04:01:01.797014 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.796958    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 04:01:01.886783 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:01:01.886756    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfed1752_5a72_44c9_ad37_86387277f9ac.slice/crio-030a347f9bafcd7db7dde1dd2217c7276003d6f55dafa2c04945007c8bc85746 WatchSource:0}: Error finding container 030a347f9bafcd7db7dde1dd2217c7276003d6f55dafa2c04945007c8bc85746: Status 404 returned error can't find the container with id 030a347f9bafcd7db7dde1dd2217c7276003d6f55dafa2c04945007c8bc85746
Nov 02 04:01:01.888211 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.888188    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 04:01:01.902499 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:01.902463    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"]
Nov 02 04:01:02.625321 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:02.625288    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj" event={"ID":"cfed1752-5a72-44c9-ad37-86387277f9ac","Type":"ContainerStarted","Data":"030a347f9bafcd7db7dde1dd2217c7276003d6f55dafa2c04945007c8bc85746"}
Nov 02 04:01:03.613384 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:03.613355    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:01:03.638704 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:03.638649    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj" event={"ID":"cfed1752-5a72-44c9-ad37-86387277f9ac","Type":"ContainerStarted","Data":"29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824"}
Nov 02 04:01:03.762120 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:03.762062    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj" podStartSLOduration=2.927830546 podStartE2EDuration="3.76204897s" totalImagesPullingTime="834.218424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:01:03.761576741 +0000 UTC m=+4407.556925305" watchObservedRunningTime="2026-11-02 04:01:03.76204897 +0000 UTC m=+4407.557397535"
Nov 02 04:01:04.644738 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:04.644698    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:05.188200 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:01:05.188166    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:01:05.898941 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:05.898907    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:10.656504 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:10.656470    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:01:11.014256 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:11.014225    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:15.909695 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:15.909657    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:20.770416 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:20.770379    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:21.396394 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:21.396359    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"]
Nov 02 04:01:21.679438 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:21.679353    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:21.844676 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:21.844636    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:25.558710 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:25.558674    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:26.397062 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.397008    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv" podUID="a1a05c6c-45d8-49bf-b6dc-4f77e40a131b" containerName="payload-pre-processing" containerID="cri-o://8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c" gracePeriod=25
Nov 02 04:01:26.676057 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.676035    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 04:01:26.703373 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.703347    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-plugins-config-volume\") pod \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\" (UID: \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\") "
Nov 02 04:01:26.703521 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.703422    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-kube-api-access-7zxgf\" (UniqueName: \"kubernetes.io/projected/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-kube-api-access-7zxgf\") pod \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\" (UID: \"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b\") "
Nov 02 04:01:26.703725 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.703704    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-plugins-config-volume" pod "a1a05c6c-45d8-49bf-b6dc-4f77e40a131b" (UID: "a1a05c6c-45d8-49bf-b6dc-4f77e40a131b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:01:26.704066 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.704050    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:01:26.705680 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.705658    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-kube-api-access-7zxgf" pod "a1a05c6c-45d8-49bf-b6dc-4f77e40a131b" (UID: "a1a05c6c-45d8-49bf-b6dc-4f77e40a131b"). InnerVolumeSpecName "kube-api-access-7zxgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:01:26.805270 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.805238    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zxgf\" (UniqueName: \"kubernetes.io/projected/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b-kube-api-access-7zxgf\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:01:26.847990 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.847954    4407 generic.go:356] "Generic (PLEG): container finished" podID="a1a05c6c-45d8-49bf-b6dc-4f77e40a131b" containerID="8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c" exitCode=0
Nov 02 04:01:26.848164 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.848003    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"
Nov 02 04:01:26.848164 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.848038    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv" event={"ID":"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b","Type":"ContainerDied","Data":"8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c"}
Nov 02 04:01:26.848164 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.848099    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv" event={"ID":"a1a05c6c-45d8-49bf-b6dc-4f77e40a131b","Type":"ContainerDied","Data":"52640206cdef8ef654e7c66b7d2a1394419c3fb3c21deb1cdbfc16805ad8e366"}
Nov 02 04:01:26.848164 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.848123    4407 scope.go:122] "RemoveContainer" containerID="8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c"
Nov 02 04:01:26.857772 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.857757    4407 scope.go:122] "RemoveContainer" containerID="8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c"
Nov 02 04:01:26.858032 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:01:26.858008    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c\": container with ID starting with 8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c not found: ID does not exist" containerID="8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c"
Nov 02 04:01:26.858086 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:26.858041    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c"} err="failed to get container status \"8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c\": rpc error: code = NotFound desc = could not find container \"8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c\": container with ID starting with 8ccb42c789c8fee9911558a830a3927df4a45f56cba62ade18b5244f60d55f7c not found: ID does not exist"
Nov 02 04:01:27.015857 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:27.015828    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"]
Nov 02 04:01:27.021438 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:27.021414    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ae1315-77b7bbd887-c5cdv"]
Nov 02 04:01:28.351386 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:28.351357    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1a05c6c-45d8-49bf-b6dc-4f77e40a131b" path="/var/lib/kubelet/pods/a1a05c6c-45d8-49bf-b6dc-4f77e40a131b/volumes"
Nov 02 04:01:30.536666 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:30.536635    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:35.190128 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:01:35.190089    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:01:35.453039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:35.452944    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:40.289056 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:40.289028    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:43.502985 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:43.502953    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:01:45.065333 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:45.065302    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:49.975313 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:49.975236    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:53.464459 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:53.464427    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:54.768320 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:54.768290    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:57.497659 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:57.497581    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"]
Nov 02 04:01:57.885997 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:57.885919    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:58.052772 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:58.052736    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:01:58.992951 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:01:58.992910    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:00.772123 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:00.772095    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:02.498848 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:02.498810    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w" podUID="27fed099-7c7c-4417-9192-4229fac3a586" containerName="payload-processing" containerID="cri-o://a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50" gracePeriod=25
Nov 02 04:02:02.758404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:02.758342    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 04:02:02.873667 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:02.873640    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27fed099-7c7c-4417-9192-4229fac3a586-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27fed099-7c7c-4417-9192-4229fac3a586-plugins-config-volume\") pod \"27fed099-7c7c-4417-9192-4229fac3a586\" (UID: \"27fed099-7c7c-4417-9192-4229fac3a586\") "
Nov 02 04:02:02.873847 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:02.873809    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27fed099-7c7c-4417-9192-4229fac3a586-kube-api-access-6sz7q\" (UniqueName: \"kubernetes.io/projected/27fed099-7c7c-4417-9192-4229fac3a586-kube-api-access-6sz7q\") pod \"27fed099-7c7c-4417-9192-4229fac3a586\" (UID: \"27fed099-7c7c-4417-9192-4229fac3a586\") "
Nov 02 04:02:02.874061 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:02.874034    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27fed099-7c7c-4417-9192-4229fac3a586-plugins-config-volume" pod "27fed099-7c7c-4417-9192-4229fac3a586" (UID: "27fed099-7c7c-4417-9192-4229fac3a586"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:02:02.876593 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:02.876559    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27fed099-7c7c-4417-9192-4229fac3a586-kube-api-access-6sz7q" pod "27fed099-7c7c-4417-9192-4229fac3a586" (UID: "27fed099-7c7c-4417-9192-4229fac3a586"). InnerVolumeSpecName "kube-api-access-6sz7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:02:02.975109 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:02.975078    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27fed099-7c7c-4417-9192-4229fac3a586-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:02.975109 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:02.975105    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sz7q\" (UniqueName: \"kubernetes.io/projected/27fed099-7c7c-4417-9192-4229fac3a586-kube-api-access-6sz7q\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:03.071383 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.071299    4407 generic.go:356] "Generic (PLEG): container finished" podID="27fed099-7c7c-4417-9192-4229fac3a586" containerID="a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50" exitCode=0
Nov 02 04:02:03.071383 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.071357    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"
Nov 02 04:02:03.071615 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.071361    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w" event={"ID":"27fed099-7c7c-4417-9192-4229fac3a586","Type":"ContainerDied","Data":"a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50"}
Nov 02 04:02:03.071615 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.071477    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w" event={"ID":"27fed099-7c7c-4417-9192-4229fac3a586","Type":"ContainerDied","Data":"28b2691c4f801b040b98f75df3b5081d2e52b04a91832979a3a6f2206381eaed"}
Nov 02 04:02:03.071615 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.071498    4407 scope.go:122] "RemoveContainer" containerID="a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50"
Nov 02 04:02:03.081420 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.081401    4407 scope.go:122] "RemoveContainer" containerID="a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50"
Nov 02 04:02:03.081695 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:02:03.081673    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50\": container with ID starting with a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50 not found: ID does not exist" containerID="a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50"
Nov 02 04:02:03.081755 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.081704    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50"} err="failed to get container status \"a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50\": rpc error: code = NotFound desc = could not find container \"a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50\": container with ID starting with a91db49e29575b7b0c5244fb01c9f94fce428eff1d37f43f1bc2b25b26246a50 not found: ID does not exist"
Nov 02 04:02:03.242425 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.242393    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"]
Nov 02 04:02:03.244958 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:03.244937    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-61f3b6-68b7d68447-7fj7w"]
Nov 02 04:02:04.351988 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:04.351956    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27fed099-7c7c-4417-9192-4229fac3a586" path="/var/lib/kubelet/pods/27fed099-7c7c-4417-9192-4229fac3a586/volumes"
Nov 02 04:02:04.996873 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:04.996834    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:05.188576 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:02:05.188540    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:02:06.541989 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:06.541958    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:06.788763 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:06.788732    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:06.951231 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:06.951150    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:07.552681 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:07.552651    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:11.245920 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:11.245890    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:12.940922 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:12.940889    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:13.128179 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:13.128147    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:13.287392 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:13.287354    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:13.586864 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:13.586786    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:13.767599 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:13.767571    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:13.959742 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:13.934072    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:18.259063 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:18.259032    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:19.642789 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:19.642757    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:19.807142 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:19.807114    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:19.908805 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:19.908718    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"]
Nov 02 04:02:19.909244 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:19.909210    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy" containerID="cri-o://70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358" gracePeriod=30
Nov 02 04:02:20.021309 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:20.011687    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:22.024082 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:22.024039    4407 patch_prober.go:29] interesting pod/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 04:02:22.024482 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:22.024125    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 04:02:23.517483 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:23.517451    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:25.186400 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.186336    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 04:02:25.216869 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.216843    4407 generic.go:356] "Generic (PLEG): container finished" podID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerID="70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358" exitCode=0
Nov 02 04:02:25.217035 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.216892    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"
Nov 02 04:02:25.217035 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.216938    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" event={"ID":"60105b4e-1c8b-4fbd-b501-5608e1392a55","Type":"ContainerDied","Data":"70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358"}
Nov 02 04:02:25.217035 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.217007    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf" event={"ID":"60105b4e-1c8b-4fbd-b501-5608e1392a55","Type":"ContainerDied","Data":"60a16c6b38f7b073489fd1b2b7f77f0a34a2e442663522bf3ab1ab60887a055a"}
Nov 02 04:02:25.217035 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.217023    4407 scope.go:122] "RemoveContainer" containerID="70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358"
Nov 02 04:02:25.226516 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.226499    4407 scope.go:122] "RemoveContainer" containerID="faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65"
Nov 02 04:02:25.235905 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.235875    4407 scope.go:122] "RemoveContainer" containerID="70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358"
Nov 02 04:02:25.236225 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:02:25.236205    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358\": container with ID starting with 70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358 not found: ID does not exist" containerID="70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358"
Nov 02 04:02:25.236282 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.236233    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358"} err="failed to get container status \"70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358\": rpc error: code = NotFound desc = could not find container \"70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358\": container with ID starting with 70c7af1528379243166f793313e9a4269f3b4ba15f975bbc97b4b7a2d2453358 not found: ID does not exist"
Nov 02 04:02:25.236282 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.236253    4407 scope.go:122] "RemoveContainer" containerID="faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65"
Nov 02 04:02:25.236516 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:02:25.236498    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65\": container with ID starting with faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65 not found: ID does not exist" containerID="faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65"
Nov 02 04:02:25.236559 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.236535    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65"} err="failed to get container status \"faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65\": rpc error: code = NotFound desc = could not find container \"faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65\": container with ID starting with faa6ec81bbbdb645de7c2be0ef4fbb90dbac91c75b16b345dd81efc5cf2eca65 not found: ID does not exist"
Nov 02 04:02:25.288391 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288356    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-credential-socket\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.288568 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288435    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-envoy\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.288568 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288509    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-podinfo\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.288568 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288557    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-data\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.288789 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288660    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-credential-socket" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:02:25.288789 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288701    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60105b4e-1c8b-4fbd-b501-5608e1392a55-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60105b4e-1c8b-4fbd-b501-5608e1392a55-istiod-ca-cert\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.288789 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288733    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-token\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-token\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.288789 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288764    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-certs\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.289024 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288837    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-socket\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.289024 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288962    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-kube-api-access-zp8w7\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-kube-api-access-zp8w7\") pod \"60105b4e-1c8b-4fbd-b501-5608e1392a55\" (UID: \"60105b4e-1c8b-4fbd-b501-5608e1392a55\") "
Nov 02 04:02:25.289024 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288837    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-data" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:02:25.289024 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.288972    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-certs" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:02:25.289234 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.289073    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60105b4e-1c8b-4fbd-b501-5608e1392a55-istiod-ca-cert" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:02:25.289234 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.289073    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-socket" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:02:25.289519 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.289449    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.289519 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.289464    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.289519 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.289474    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60105b4e-1c8b-4fbd-b501-5608e1392a55-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.289519 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.289482    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.289519 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.289490    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.291055 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.291022    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-podinfo" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:02:25.291055 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.291047    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-envoy" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:02:25.291258 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.291214    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-token" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:02:25.291319 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.291263    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-kube-api-access-zp8w7" pod "60105b4e-1c8b-4fbd-b501-5608e1392a55" (UID: "60105b4e-1c8b-4fbd-b501-5608e1392a55"). InnerVolumeSpecName "kube-api-access-zp8w7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:02:25.302468 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.302447    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:25.397967 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.390785    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.398438 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.398411    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.398645 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.398606    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.398786 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.398769    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp8w7\" (UniqueName: \"kubernetes.io/projected/60105b4e-1c8b-4fbd-b501-5608e1392a55-kube-api-access-zp8w7\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:02:25.639424 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.639387    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:02:25.642306 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.642279    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"]
Nov 02 04:02:25.644533 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:25.644510    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-9b7f43-openshift-default-5c6c9c86b9-cz4tf"]
Nov 02 04:02:26.353387 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:26.353350    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" path="/var/lib/kubelet/pods/60105b4e-1c8b-4fbd-b501-5608e1392a55/volumes"
Nov 02 04:02:27.733162 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:27.733123    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:27.898320 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:27.898272    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:28.699345 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:28.699305    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:33.654044 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:33.654013    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:33.872684 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:33.872640    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:34.036092 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:34.036052    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:35.189811 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:02:35.189776    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:02:37.838795 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:37.838758    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:38.597588 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:38.597560    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:38.784198 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:38.784167    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:39.426328 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:39.426296    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:39.666717 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:39.666688    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:39.826141 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:39.826109    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:40.338229 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:40.338192    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:43.843110 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:43.843074    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:44.776904 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:44.776872    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:46.228404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:46.228357    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:46.461305 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:46.461272    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:46.659815 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:46.659786    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:49.122286 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:49.122258    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:50.751909 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:50.751875    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:51.024080 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:51.024004    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:51.185090 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:51.185058    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:51.657928 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:51.657895    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:53.821531 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:53.821487    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:55.961915 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:55.961880    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:57.271394 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:57.271357    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:57.433840 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:57.433806    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:57.604309 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:57.604229    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:58.887848 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:58.887813    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"]
Nov 02 04:02:58.996691 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:58.996653    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"]
Nov 02 04:02:59.272295 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:59.272260    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:02:59.433454 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:02:59.433409    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:01.017390 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:01.017361    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:01.025714 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:01.025683    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:03:03.888931 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:03.888882    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" podUID="ad206f87-2062-48a7-b2c6-df915c4a7a63" containerName="payload-processing" containerID="cri-o://4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481" gracePeriod=25
Nov 02 04:03:03.996956 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:03.996921    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" podUID="1cff9b9b-3a83-4704-a39a-c39f505d8686" containerName="payload-pre-processing" containerID="cri-o://7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb" gracePeriod=25
Nov 02 04:03:04.059680 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:04.059645    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" podUID="1cff9b9b-3a83-4704-a39a-c39f505d8686" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:03:04.059875 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:04.059616    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" podUID="ad206f87-2062-48a7-b2c6-df915c4a7a63" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:03:05.188708 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:03:05.188673    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:03:06.061529 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:06.061475    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:10.814582 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:10.814536    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:13.828881 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:13.828830    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:13.978340 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:13.978311    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:14.059376 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:14.059335    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" podUID="1cff9b9b-3a83-4704-a39a-c39f505d8686" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:03:14.059603 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:14.059376    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" podUID="ad206f87-2062-48a7-b2c6-df915c4a7a63" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:03:15.541610 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:15.541575    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:20.361633 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:20.361522    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:24.058843 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:03:24.058816    4407 logging.go:55] [core] [Channel #2733 SubChannel #2735]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.119:50052", ServerName: "10.133.0.119:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.119:50052: connect: connection refused"
Nov 02 04:03:24.058843 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:03:24.058816    4407 logging.go:55] [core] [Channel #2732 SubChannel #2734]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.120:50052", ServerName: "10.133.0.120:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.120:50052: connect: connection refused"
Nov 02 04:03:24.178942 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.178916    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 04:03:24.279160 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.279133    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 04:03:24.298180 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.298151    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad206f87-2062-48a7-b2c6-df915c4a7a63-kube-api-access-jv4mz\" (UniqueName: \"kubernetes.io/projected/ad206f87-2062-48a7-b2c6-df915c4a7a63-kube-api-access-jv4mz\") pod \"ad206f87-2062-48a7-b2c6-df915c4a7a63\" (UID: \"ad206f87-2062-48a7-b2c6-df915c4a7a63\") "
Nov 02 04:03:24.298339 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.298238    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad206f87-2062-48a7-b2c6-df915c4a7a63-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad206f87-2062-48a7-b2c6-df915c4a7a63-plugins-config-volume\") pod \"ad206f87-2062-48a7-b2c6-df915c4a7a63\" (UID: \"ad206f87-2062-48a7-b2c6-df915c4a7a63\") "
Nov 02 04:03:24.298612 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.298587    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad206f87-2062-48a7-b2c6-df915c4a7a63-plugins-config-volume" pod "ad206f87-2062-48a7-b2c6-df915c4a7a63" (UID: "ad206f87-2062-48a7-b2c6-df915c4a7a63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:03:24.300487 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.300467    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad206f87-2062-48a7-b2c6-df915c4a7a63-kube-api-access-jv4mz" pod "ad206f87-2062-48a7-b2c6-df915c4a7a63" (UID: "ad206f87-2062-48a7-b2c6-df915c4a7a63"). InnerVolumeSpecName "kube-api-access-jv4mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:03:24.399932 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.399902    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cff9b9b-3a83-4704-a39a-c39f505d8686-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cff9b9b-3a83-4704-a39a-c39f505d8686-plugins-config-volume\") pod \"1cff9b9b-3a83-4704-a39a-c39f505d8686\" (UID: \"1cff9b9b-3a83-4704-a39a-c39f505d8686\") "
Nov 02 04:03:24.400091 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.400067    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cff9b9b-3a83-4704-a39a-c39f505d8686-kube-api-access-p2rmr\" (UniqueName: \"kubernetes.io/projected/1cff9b9b-3a83-4704-a39a-c39f505d8686-kube-api-access-p2rmr\") pod \"1cff9b9b-3a83-4704-a39a-c39f505d8686\" (UID: \"1cff9b9b-3a83-4704-a39a-c39f505d8686\") "
Nov 02 04:03:24.400289 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.400241    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cff9b9b-3a83-4704-a39a-c39f505d8686-plugins-config-volume" pod "1cff9b9b-3a83-4704-a39a-c39f505d8686" (UID: "1cff9b9b-3a83-4704-a39a-c39f505d8686"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:03:24.400477 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.400445    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv4mz\" (UniqueName: \"kubernetes.io/projected/ad206f87-2062-48a7-b2c6-df915c4a7a63-kube-api-access-jv4mz\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:24.400561 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.400485    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad206f87-2062-48a7-b2c6-df915c4a7a63-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:24.400561 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.400508    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cff9b9b-3a83-4704-a39a-c39f505d8686-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:24.402327 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.402306    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cff9b9b-3a83-4704-a39a-c39f505d8686-kube-api-access-p2rmr" pod "1cff9b9b-3a83-4704-a39a-c39f505d8686" (UID: "1cff9b9b-3a83-4704-a39a-c39f505d8686"). InnerVolumeSpecName "kube-api-access-p2rmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:03:24.501503 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.501464    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2rmr\" (UniqueName: \"kubernetes.io/projected/1cff9b9b-3a83-4704-a39a-c39f505d8686-kube-api-access-p2rmr\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:24.566591 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.566552    4407 generic.go:356] "Generic (PLEG): container finished" podID="1cff9b9b-3a83-4704-a39a-c39f505d8686" containerID="7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb" exitCode=0
Nov 02 04:03:24.566802 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.566603    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"
Nov 02 04:03:24.566802 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.566654    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" event={"ID":"1cff9b9b-3a83-4704-a39a-c39f505d8686","Type":"ContainerDied","Data":"7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb"}
Nov 02 04:03:24.566802 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.566713    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" event={"ID":"1cff9b9b-3a83-4704-a39a-c39f505d8686","Type":"ContainerDied","Data":"15d8e34d94f58a6191458c5a1e942fe371add7866368d31efb68900cf2998e42"}
Nov 02 04:03:24.566802 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.566740    4407 scope.go:122] "RemoveContainer" containerID="7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb"
Nov 02 04:03:24.568037 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.568011    4407 generic.go:356] "Generic (PLEG): container finished" podID="ad206f87-2062-48a7-b2c6-df915c4a7a63" containerID="4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481" exitCode=0
Nov 02 04:03:24.568144 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.568060    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"
Nov 02 04:03:24.568144 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.568088    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" event={"ID":"ad206f87-2062-48a7-b2c6-df915c4a7a63","Type":"ContainerDied","Data":"4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481"}
Nov 02 04:03:24.568144 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.568121    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" event={"ID":"ad206f87-2062-48a7-b2c6-df915c4a7a63","Type":"ContainerDied","Data":"86a92a10ff9c59cd9e5d1a8c4dba60293fde6dfdebc164fcbd5f6c6036a25f5c"}
Nov 02 04:03:24.577176 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.577158    4407 scope.go:122] "RemoveContainer" containerID="7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb"
Nov 02 04:03:24.577437 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:03:24.577419    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb\": container with ID starting with 7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb not found: ID does not exist" containerID="7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb"
Nov 02 04:03:24.577523 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.577445    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb"} err="failed to get container status \"7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb\": rpc error: code = NotFound desc = could not find container \"7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb\": container with ID starting with 7209cc8b7ca0bd10e33eb5f42ab10770bfe8464f7cdc11d268b518e2cc5f76eb not found: ID does not exist"
Nov 02 04:03:24.577523 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.577462    4407 scope.go:122] "RemoveContainer" containerID="4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481"
Nov 02 04:03:24.586357 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.586339    4407 scope.go:122] "RemoveContainer" containerID="4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481"
Nov 02 04:03:24.586661 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:03:24.586641    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481\": container with ID starting with 4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481 not found: ID does not exist" containerID="4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481"
Nov 02 04:03:24.586733 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.586668    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481"} err="failed to get container status \"4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481\": rpc error: code = NotFound desc = could not find container \"4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481\": container with ID starting with 4402e2904fcb36146d776bb5bb1fe9e44e744613f12f38c2170610774313d481 not found: ID does not exist"
Nov 02 04:03:24.731736 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.731650    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"]
Nov 02 04:03:24.735073 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.735051    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd"]
Nov 02 04:03:24.845410 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.845374    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"]
Nov 02 04:03:24.849406 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:24.849375    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6"]
Nov 02 04:03:25.058727 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:25.058686    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-147104-7c757b5c59-lr4pd" podUID="1cff9b9b-3a83-4704-a39a-c39f505d8686" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.119:50052\" within 1s: context deadline exceeded"
Nov 02 04:03:25.058942 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:25.058684    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-147104-869847c6d9-6tsj6" podUID="ad206f87-2062-48a7-b2c6-df915c4a7a63" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.120:50052\" within 1s: context deadline exceeded"
Nov 02 04:03:25.523124 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:25.523081    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:25.995066 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:25.995032    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:26.214417 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:26.214380    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:26.351879 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:26.351787    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cff9b9b-3a83-4704-a39a-c39f505d8686" path="/var/lib/kubelet/pods/1cff9b9b-3a83-4704-a39a-c39f505d8686/volumes"
Nov 02 04:03:26.352311 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:26.352291    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad206f87-2062-48a7-b2c6-df915c4a7a63" path="/var/lib/kubelet/pods/ad206f87-2062-48a7-b2c6-df915c4a7a63/volumes"
Nov 02 04:03:26.372760 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:26.372732    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:26.898890 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:26.898857    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:31.198504 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:31.198470    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:33.121380 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:33.121347    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:33.287120 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:33.287085    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:33.360476 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:33.360438    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"]
Nov 02 04:03:33.360686 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:33.360663    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy" containerID="cri-o://98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22" gracePeriod=30
Nov 02 04:03:33.473248 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:33.473163    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:35.190337 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:03:35.190295    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:03:35.229915 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.229885    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"]
Nov 02 04:03:35.230090 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.230051    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts" podUID="4910eacd-0342-4cd2-ad5d-fc5ead694dbb" containerName="maas-api" containerID="cri-o://338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e" gracePeriod=30
Nov 02 04:03:35.346948 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.346915    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:35.500346 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.500321    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 04:03:35.637966 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.637931    4407 generic.go:356] "Generic (PLEG): container finished" podID="4910eacd-0342-4cd2-ad5d-fc5ead694dbb" containerID="338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e" exitCode=0
Nov 02 04:03:35.638149 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.637981    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"
Nov 02 04:03:35.638149 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.638015    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts" event={"ID":"4910eacd-0342-4cd2-ad5d-fc5ead694dbb","Type":"ContainerDied","Data":"338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e"}
Nov 02 04:03:35.638149 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.638070    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts" event={"ID":"4910eacd-0342-4cd2-ad5d-fc5ead694dbb","Type":"ContainerDied","Data":"f4c68dbbfbd426bc38a3d78886c6b25f44e1aa0128cca470940c2287157da402"}
Nov 02 04:03:35.638149 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.638103    4407 scope.go:122] "RemoveContainer" containerID="338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e"
Nov 02 04:03:35.647917 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.647882    4407 scope.go:122] "RemoveContainer" containerID="338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e"
Nov 02 04:03:35.648220 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:03:35.648198    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e\": container with ID starting with 338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e not found: ID does not exist" containerID="338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e"
Nov 02 04:03:35.648323 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.648232    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e"} err="failed to get container status \"338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e\": rpc error: code = NotFound desc = could not find container \"338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e\": container with ID starting with 338c80f4ecfe372a87d41075076ff7ef6d68e44426e78656e796e11eb273f26e not found: ID does not exist"
Nov 02 04:03:35.669176 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.669149    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls\") pod \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") "
Nov 02 04:03:35.669377 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.669267    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-metrics-certs\") pod \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") "
Nov 02 04:03:35.669377 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.669360    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-kube-api-access-fcw8q\" (UniqueName: \"kubernetes.io/projected/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-kube-api-access-fcw8q\") pod \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") "
Nov 02 04:03:35.669482 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.669432    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-openshift-service-ca\") pod \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\" (UID: \"4910eacd-0342-4cd2-ad5d-fc5ead694dbb\") "
Nov 02 04:03:35.669900 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.669870    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-openshift-service-ca" pod "4910eacd-0342-4cd2-ad5d-fc5ead694dbb" (UID: "4910eacd-0342-4cd2-ad5d-fc5ead694dbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:03:35.670032 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.670017    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:35.671513 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.671488    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls" pod "4910eacd-0342-4cd2-ad5d-fc5ead694dbb" (UID: "4910eacd-0342-4cd2-ad5d-fc5ead694dbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:03:35.672053 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.672031    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-metrics-certs" pod "4910eacd-0342-4cd2-ad5d-fc5ead694dbb" (UID: "4910eacd-0342-4cd2-ad5d-fc5ead694dbb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:03:35.672142 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.672094    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-kube-api-access-fcw8q" pod "4910eacd-0342-4cd2-ad5d-fc5ead694dbb" (UID: "4910eacd-0342-4cd2-ad5d-fc5ead694dbb"). InnerVolumeSpecName "kube-api-access-fcw8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:03:35.776649 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.776528    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:35.776649 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.776566    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:35.776649 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:35.776582    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcw8q\" (UniqueName: \"kubernetes.io/projected/4910eacd-0342-4cd2-ad5d-fc5ead694dbb-kube-api-access-fcw8q\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:36.060338 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:36.060259    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"]
Nov 02 04:03:36.110988 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:36.110954    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4c3ea5-6b8b85fcc8-cr5ts"]
Nov 02 04:03:36.360537 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:36.360446    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4910eacd-0342-4cd2-ad5d-fc5ead694dbb" path="/var/lib/kubelet/pods/4910eacd-0342-4cd2-ad5d-fc5ead694dbb/volumes"
Nov 02 04:03:36.986386 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:36.986349    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:37.197543 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:37.197509    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:37.358551 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:37.358517    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:37.869398 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:37.869364    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:38.641010 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.640984    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 04:03:38.658176 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.658142    4407 generic.go:356] "Generic (PLEG): container finished" podID="dfb3623e-4997-4231-9052-54b65d6441f1" containerID="98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22" exitCode=0
Nov 02 04:03:38.658287 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.658187    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"
Nov 02 04:03:38.658287 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.658232    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" event={"ID":"dfb3623e-4997-4231-9052-54b65d6441f1","Type":"ContainerDied","Data":"98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22"}
Nov 02 04:03:38.658287 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.658283    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9" event={"ID":"dfb3623e-4997-4231-9052-54b65d6441f1","Type":"ContainerDied","Data":"0e1d8a9a1f01225684f7a4454a8fb7576635ab5ac72315820fc29ff8da828f99"}
Nov 02 04:03:38.658405 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.658305    4407 scope.go:122] "RemoveContainer" containerID="98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22"
Nov 02 04:03:38.667644 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.667611    4407 scope.go:122] "RemoveContainer" containerID="8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d"
Nov 02 04:03:38.676122 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.676100    4407 scope.go:122] "RemoveContainer" containerID="98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22"
Nov 02 04:03:38.676389 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:03:38.676370    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22\": container with ID starting with 98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22 not found: ID does not exist" containerID="98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22"
Nov 02 04:03:38.676445 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.676399    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22"} err="failed to get container status \"98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22\": rpc error: code = NotFound desc = could not find container \"98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22\": container with ID starting with 98138f0bcfd5ddde6f7105591ef9b76a87af850a1eeade9fa0360007e5f7bd22 not found: ID does not exist"
Nov 02 04:03:38.676445 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.676419    4407 scope.go:122] "RemoveContainer" containerID="8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d"
Nov 02 04:03:38.676749 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:03:38.676730    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d\": container with ID starting with 8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d not found: ID does not exist" containerID="8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d"
Nov 02 04:03:38.676796 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.676758    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d"} err="failed to get container status \"8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d\": rpc error: code = NotFound desc = could not find container \"8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d\": container with ID starting with 8dce4dcb76541028b2dbeff8bee08008c2fa312af8676df1aac585fdfd25393d not found: ID does not exist"
Nov 02 04:03:38.713544 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713519    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-data\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.713690 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713564    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-istio-token\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-istio-token\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.713690 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713597    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dfb3623e-4997-4231-9052-54b65d6441f1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb3623e-4997-4231-9052-54b65d6441f1-istiod-ca-cert\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.713690 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713650    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-certs\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.713858 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713721    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-envoy\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.713858 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713745    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-kube-api-access-5bkjk\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-kube-api-access-5bkjk\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.713858 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713794    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dfb3623e-4997-4231-9052-54b65d6441f1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb3623e-4997-4231-9052-54b65d6441f1-istio-podinfo\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.713858 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713829    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-data" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:03:38.714069 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713865    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-credential-socket\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.714069 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.713955    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-socket\") pod \"dfb3623e-4997-4231-9052-54b65d6441f1\" (UID: \"dfb3623e-4997-4231-9052-54b65d6441f1\") "
Nov 02 04:03:38.714069 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714019    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-certs" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:03:38.714069 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714038    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfb3623e-4997-4231-9052-54b65d6441f1-istiod-ca-cert" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:03:38.714280 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714230    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-credential-socket" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:03:38.714449 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714394    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-socket" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:03:38.714681 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714657    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:38.714758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714686    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:38.714758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714703    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:38.714758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714717    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb3623e-4997-4231-9052-54b65d6441f1-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:38.714758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.714731    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:38.715986 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.715962    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-istio-token" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:03:38.716085 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.716057    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-kube-api-access-5bkjk" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "kube-api-access-5bkjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:03:38.716143 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.716123    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dfb3623e-4997-4231-9052-54b65d6441f1-istio-podinfo" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:03:38.716186 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.716166    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-envoy" pod "dfb3623e-4997-4231-9052-54b65d6441f1" (UID: "dfb3623e-4997-4231-9052-54b65d6441f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:03:38.815760 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.815726    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:38.815760 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.815752    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb3623e-4997-4231-9052-54b65d6441f1-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:38.815760 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.815764    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bkjk\" (UniqueName: \"kubernetes.io/projected/dfb3623e-4997-4231-9052-54b65d6441f1-kube-api-access-5bkjk\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:38.816042 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:38.815773    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb3623e-4997-4231-9052-54b65d6441f1-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:39.084973 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:39.084942    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"]
Nov 02 04:03:39.090847 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:39.090824    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-61f3b6-openshift-default-c7b4bfbfc-cw2x9"]
Nov 02 04:03:40.352493 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:40.352459    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" path="/var/lib/kubelet/pods/dfb3623e-4997-4231-9052-54b65d6441f1/volumes"
Nov 02 04:03:42.409244 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:42.409212    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:43.492074 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:43.492039    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:43.647407 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:43.647373    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:43.812287 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:43.812254    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:47.389161 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:47.389130    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:47.551178 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:47.551127    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:47.709824 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:47.709737    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:48.257533 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:48.257501    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:03:51.038008 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:51.037977    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:52.084962 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.084925    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"]
Nov 02 04:03:52.085403 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.085185    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj" podUID="cfed1752-5a72-44c9-ad37-86387277f9ac" containerName="maas-api" containerID="cri-o://29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824" gracePeriod=30
Nov 02 04:03:52.205788 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.205761    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:52.366697 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.366667    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:03:52.507784 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.507751    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls\") pod \"cfed1752-5a72-44c9-ad37-86387277f9ac\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") "
Nov 02 04:03:52.507981 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.507876    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-metrics-certs\") pod \"cfed1752-5a72-44c9-ad37-86387277f9ac\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") "
Nov 02 04:03:52.507981 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.507919    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfed1752-5a72-44c9-ad37-86387277f9ac-kube-api-access-65lsp\" (UniqueName: \"kubernetes.io/projected/cfed1752-5a72-44c9-ad37-86387277f9ac-kube-api-access-65lsp\") pod \"cfed1752-5a72-44c9-ad37-86387277f9ac\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") "
Nov 02 04:03:52.508122 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.508074    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfed1752-5a72-44c9-ad37-86387277f9ac-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfed1752-5a72-44c9-ad37-86387277f9ac-openshift-service-ca\") pod \"cfed1752-5a72-44c9-ad37-86387277f9ac\" (UID: \"cfed1752-5a72-44c9-ad37-86387277f9ac\") "
Nov 02 04:03:52.508552 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.508514    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfed1752-5a72-44c9-ad37-86387277f9ac-openshift-service-ca" pod "cfed1752-5a72-44c9-ad37-86387277f9ac" (UID: "cfed1752-5a72-44c9-ad37-86387277f9ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:03:52.510104 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.510073    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls" pod "cfed1752-5a72-44c9-ad37-86387277f9ac" (UID: "cfed1752-5a72-44c9-ad37-86387277f9ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:03:52.510731 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.510702    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-metrics-certs" pod "cfed1752-5a72-44c9-ad37-86387277f9ac" (UID: "cfed1752-5a72-44c9-ad37-86387277f9ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:03:52.510828 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.510709    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfed1752-5a72-44c9-ad37-86387277f9ac-kube-api-access-65lsp" pod "cfed1752-5a72-44c9-ad37-86387277f9ac" (UID: "cfed1752-5a72-44c9-ad37-86387277f9ac"). InnerVolumeSpecName "kube-api-access-65lsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:03:52.609835 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.609753    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:52.609835 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.609781    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfed1752-5a72-44c9-ad37-86387277f9ac-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:52.609835 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.609792    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65lsp\" (UniqueName: \"kubernetes.io/projected/cfed1752-5a72-44c9-ad37-86387277f9ac-kube-api-access-65lsp\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:52.609835 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.609801    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfed1752-5a72-44c9-ad37-86387277f9ac-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:03:52.736527 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.736496    4407 generic.go:356] "Generic (PLEG): container finished" podID="cfed1752-5a72-44c9-ad37-86387277f9ac" containerID="29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824" exitCode=0
Nov 02 04:03:52.736725 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.736546    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"
Nov 02 04:03:52.736725 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.736583    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj" event={"ID":"cfed1752-5a72-44c9-ad37-86387277f9ac","Type":"ContainerDied","Data":"29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824"}
Nov 02 04:03:52.736725 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.736675    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj" event={"ID":"cfed1752-5a72-44c9-ad37-86387277f9ac","Type":"ContainerDied","Data":"030a347f9bafcd7db7dde1dd2217c7276003d6f55dafa2c04945007c8bc85746"}
Nov 02 04:03:52.736725 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.736704    4407 scope.go:122] "RemoveContainer" containerID="29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824"
Nov 02 04:03:52.747527 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.747512    4407 scope.go:122] "RemoveContainer" containerID="29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824"
Nov 02 04:03:52.747822 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:03:52.747805    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824\": container with ID starting with 29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824 not found: ID does not exist" containerID="29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824"
Nov 02 04:03:52.747887 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.747830    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824"} err="failed to get container status \"29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824\": rpc error: code = NotFound desc = could not find container \"29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824\": container with ID starting with 29ce43a274fba740ada133f80561c6b58e10692cb69d6889d65c640b07ea6824 not found: ID does not exist"
Nov 02 04:03:52.902932 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.902846    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"]
Nov 02 04:03:52.905221 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:52.905199    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e69cf2-67c86d8dcd-p8mrj"]
Nov 02 04:03:53.424778 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:53.424752    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:53.783326 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:53.783297    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:54.135252 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:54.135130    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:54.299528 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:54.299474    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:54.351187 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:54.351152    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfed1752-5a72-44c9-ad37-86387277f9ac" path="/var/lib/kubelet/pods/cfed1752-5a72-44c9-ad37-86387277f9ac/volumes"
Nov 02 04:03:54.595581 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:54.595549    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:54.766265 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:54.766222    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:54.980602 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:54.980488    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:55.313261 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:55.313226    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:55.785087 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:55.785040    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:55.955510 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:55.955476    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:56.380032 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:56.379988    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:56.703775 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:56.703661    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:57.229829 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:57.229775    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:57.384583 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:57.384538    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:57.542162 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:57.542107    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:57.706017 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:57.705957    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:57.862036 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:57.861901    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:03:58.351956 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:03:58.351886    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:00.427316 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:00.427284    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:00.592788 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:00.592758    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:00.760268 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:00.760244    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:01.743565 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:01.743533    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:01.909501 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:01.909459    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:02.038280 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:02.038203    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"]
Nov 02 04:04:02.038447 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:02.038421    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy" containerID="cri-o://200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f" gracePeriod=30
Nov 02 04:04:02.083120 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:02.083090    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:02.482561 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:02.482483    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:02.987815 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:02.987782    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:03.196846 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:03.196811    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:03.395748 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:03.395710    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:03.558693 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:03.558657    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:03.834687 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:03.834654    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"]
Nov 02 04:04:03.834930 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:03.834908    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy" containerID="cri-o://19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72" gracePeriod=30
Nov 02 04:04:03.847077 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:03.847054    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:04.278832 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:04.278804    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:04.809675 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:04.809644    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:05.015098 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:05.015043    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:05.185151 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:05.185058    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:05.206515 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:05.206480    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:04:05.904115 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:05.904080    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 04:04:05.904523 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:05.904141    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 04:04:06.428436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:06.428398    4407 patch_prober.go:29] interesting pod/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 04:04:06.428605 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:06.428482    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 04:04:06.946389 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:06.946361    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:07.114999 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.114969    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:07.309667 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.309642    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 04:04:07.427933 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.427900    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-token\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-token\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.427985    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-data\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428024    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-kube-api-access-tck2t\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-kube-api-access-tck2t\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428089    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-envoy\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428265 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428146    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-socket\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428265 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428221    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ac08935-e676-4ffc-ab77-4abc80198b6c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac08935-e676-4ffc-ab77-4abc80198b6c-istiod-ca-cert\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428355 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428283    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-certs\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428355 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428317    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-podinfo\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428475 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428355    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-data" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:07.428475 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428377    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-credential-socket\") pod \"6ac08935-e676-4ffc-ab77-4abc80198b6c\" (UID: \"6ac08935-e676-4ffc-ab77-4abc80198b6c\") "
Nov 02 04:04:07.428590 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428535    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-certs" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:07.428815 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428696    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac08935-e676-4ffc-ab77-4abc80198b6c-istiod-ca-cert" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:04:07.428815 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.428770    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-credential-socket" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:07.429057 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.429034    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-socket" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:07.429217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.429198    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac08935-e676-4ffc-ab77-4abc80198b6c-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.429268 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.429225    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.429268 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.429242    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.429268 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.429258    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.430498 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.430476    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-envoy" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:07.430498 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.430489    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-podinfo" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:04:07.430827 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.430810    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-token" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:07.430880 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.430833    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-kube-api-access-tck2t" pod "6ac08935-e676-4ffc-ab77-4abc80198b6c" (UID: "6ac08935-e676-4ffc-ab77-4abc80198b6c"). InnerVolumeSpecName "kube-api-access-tck2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:07.530987 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.530953    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.530987 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.530983    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.530987 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.530993    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tck2t\" (UniqueName: \"kubernetes.io/projected/6ac08935-e676-4ffc-ab77-4abc80198b6c-kube-api-access-tck2t\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.531248 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.531004    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.531248 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.531015    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac08935-e676-4ffc-ab77-4abc80198b6c-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:07.866870 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.866773    4407 generic.go:356] "Generic (PLEG): container finished" podID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerID="200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f" exitCode=0
Nov 02 04:04:07.866870 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.866837    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" event={"ID":"6ac08935-e676-4ffc-ab77-4abc80198b6c","Type":"ContainerDied","Data":"200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f"}
Nov 02 04:04:07.867112 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.866890    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"
Nov 02 04:04:07.867112 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.866890    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l" event={"ID":"6ac08935-e676-4ffc-ab77-4abc80198b6c","Type":"ContainerDied","Data":"42861ce3c531ac1f80bd52f8ac71da8fec9296e382c7e68fbcc76400ba45c46a"}
Nov 02 04:04:07.867112 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.866929    4407 scope.go:122] "RemoveContainer" containerID="200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f"
Nov 02 04:04:07.884953 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.884931    4407 scope.go:122] "RemoveContainer" containerID="1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc"
Nov 02 04:04:07.894257 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.894238    4407 scope.go:122] "RemoveContainer" containerID="200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f"
Nov 02 04:04:07.894562 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:07.894542    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f\": container with ID starting with 200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f not found: ID does not exist" containerID="200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f"
Nov 02 04:04:07.894668 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.894572    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f"} err="failed to get container status \"200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f\": rpc error: code = NotFound desc = could not find container \"200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f\": container with ID starting with 200b3c02f020321fada81db11b20d21b3d8b188bbf29ab9f72c054526605028f not found: ID does not exist"
Nov 02 04:04:07.894668 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.894595    4407 scope.go:122] "RemoveContainer" containerID="1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc"
Nov 02 04:04:07.894899 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:07.894878    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc\": container with ID starting with 1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc not found: ID does not exist" containerID="1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc"
Nov 02 04:04:07.894943 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:07.894906    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc"} err="failed to get container status \"1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc\": rpc error: code = NotFound desc = could not find container \"1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc\": container with ID starting with 1884f547a70bf013c1dcee87789f2795e82044b17af2d63d87bdcfc626e5c4cc not found: ID does not exist"
Nov 02 04:04:08.034341 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:08.034309    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"]
Nov 02 04:04:08.084945 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:08.084912    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-a7cf93-openshift-default-c59cc76df-m8p7l"]
Nov 02 04:04:08.258070 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:08.258045    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:08.351415 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:08.351381    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" path="/var/lib/kubelet/pods/6ac08935-e676-4ffc-ab77-4abc80198b6c/volumes"
Nov 02 04:04:09.095100 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.095073    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 04:04:09.224779 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.224678    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"]
Nov 02 04:04:09.240651 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.240601    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"]
Nov 02 04:04:09.251048 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251022    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istiod-ca-cert\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251053    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-credential-socket\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251115    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-envoy\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251265    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-kube-api-access-q7k9m\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-kube-api-access-q7k9m\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251297    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-socket\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251353    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-podinfo\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251533 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251369    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-credential-socket" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:09.251533 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251384    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-token\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-token\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251533 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251440    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istiod-ca-cert" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:04:09.251533 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251454    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-certs\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251533 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251485    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-data\") pod \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\" (UID: \"ebf9995a-bd67-4765-b6d8-0e1cfeced751\") "
Nov 02 04:04:09.251843 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251601    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-socket" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:09.251911 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.251893    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-data" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:09.252243 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.252130    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.252364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.252252    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.252364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.252269    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.252364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.252282    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.252364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.252333    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-certs" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:09.253799 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.253772    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-token" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:09.254011 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.253990    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-podinfo" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:04:09.254067 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.254000    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-envoy" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:09.254312 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.254295    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-kube-api-access-q7k9m" pod "ebf9995a-bd67-4765-b6d8-0e1cfeced751" (UID: "ebf9995a-bd67-4765-b6d8-0e1cfeced751"). InnerVolumeSpecName "kube-api-access-q7k9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:09.353693 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.353534    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7k9m\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-kube-api-access-q7k9m\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.353693 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.353569    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.353693 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.353585    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.353693 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.353601    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.353693 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.353614    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf9995a-bd67-4765-b6d8-0e1cfeced751-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:09.364650 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.364612    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:09.881859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.881819    4407 generic.go:356] "Generic (PLEG): container finished" podID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerID="19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72" exitCode=0
Nov 02 04:04:09.882065 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.881855    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" event={"ID":"ebf9995a-bd67-4765-b6d8-0e1cfeced751","Type":"ContainerDied","Data":"19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72"}
Nov 02 04:04:09.882065 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.881911    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w" event={"ID":"ebf9995a-bd67-4765-b6d8-0e1cfeced751","Type":"ContainerDied","Data":"a62bb20e58705cdd029012e267a0b2f9396424008f1ee525a0bbaa33f96f93f7"}
Nov 02 04:04:09.882065 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.881935    4407 scope.go:122] "RemoveContainer" containerID="19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72"
Nov 02 04:04:09.882065 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.881868    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"
Nov 02 04:04:09.891787 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.891772    4407 scope.go:122] "RemoveContainer" containerID="26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4"
Nov 02 04:04:09.901119 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.901100    4407 scope.go:122] "RemoveContainer" containerID="19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72"
Nov 02 04:04:09.901398 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:09.901373    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72\": container with ID starting with 19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72 not found: ID does not exist" containerID="19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72"
Nov 02 04:04:09.901532 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.901408    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72"} err="failed to get container status \"19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72\": rpc error: code = NotFound desc = could not find container \"19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72\": container with ID starting with 19530ff000783c5e4f4f5a7fbb30532313bd813ba6495d6c72195abb8ba73b72 not found: ID does not exist"
Nov 02 04:04:09.901532 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.901430    4407 scope.go:122] "RemoveContainer" containerID="26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4"
Nov 02 04:04:09.901763 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:09.901742    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4\": container with ID starting with 26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4 not found: ID does not exist" containerID="26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4"
Nov 02 04:04:09.901824 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:09.901772    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4"} err="failed to get container status \"26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4\": rpc error: code = NotFound desc = could not find container \"26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4\": container with ID starting with 26330b03d38c2b54dfbc31b79f5bb026f068c662c6a8f83558e8247ecd1da1c4 not found: ID does not exist"
Nov 02 04:04:10.050797 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.050762    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"]
Nov 02 04:04:10.070937 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.070902    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:10.101135 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.101104    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-147104-openshift-default-7c49449cb7-skk7w"]
Nov 02 04:04:10.258532 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.258484    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:10.359895 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.359858    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" path="/var/lib/kubelet/pods/ebf9995a-bd67-4765-b6d8-0e1cfeced751/volumes"
Nov 02 04:04:10.454788 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.454755    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"]
Nov 02 04:04:10.455304 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455285    4407 cpu_manager.go:414] "containerMap: removing container" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455306    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455321    4407 cpu_manager.go:414] "containerMap: removing container" podUID="27fed099-7c7c-4417-9192-4229fac3a586" containerName="payload-processing"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455329    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27fed099-7c7c-4417-9192-4229fac3a586" containerName="payload-processing"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455354    4407 cpu_manager.go:414] "containerMap: removing container" podUID="ad206f87-2062-48a7-b2c6-df915c4a7a63" containerName="payload-processing"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455363    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad206f87-2062-48a7-b2c6-df915c4a7a63" containerName="payload-processing"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455375    4407 cpu_manager.go:414] "containerMap: removing container" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455382    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455399    4407 cpu_manager.go:414] "containerMap: removing container" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy"
Nov 02 04:04:10.455404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455406    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455420    4407 cpu_manager.go:414] "containerMap: removing container" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455427    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455441    4407 cpu_manager.go:414] "containerMap: removing container" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455449    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455464    4407 cpu_manager.go:414] "containerMap: removing container" podUID="4910eacd-0342-4cd2-ad5d-fc5ead694dbb" containerName="maas-api"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455471    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4910eacd-0342-4cd2-ad5d-fc5ead694dbb" containerName="maas-api"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455481    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a1a05c6c-45d8-49bf-b6dc-4f77e40a131b" containerName="payload-pre-processing"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455489    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1a05c6c-45d8-49bf-b6dc-4f77e40a131b" containerName="payload-pre-processing"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455513    4407 cpu_manager.go:414] "containerMap: removing container" podUID="cfed1752-5a72-44c9-ad37-86387277f9ac" containerName="maas-api"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455521    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfed1752-5a72-44c9-ad37-86387277f9ac" containerName="maas-api"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455542    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455549    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7116adfc-c3b3-47f5-949d-148d52c37b5a" containerName="cleanup"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455560    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455566    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd6c1b51-5a41-4c74-b01d-5bdc57d662ac" containerName="cleanup"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455587    4407 cpu_manager.go:414] "containerMap: removing container" podUID="1cff9b9b-3a83-4704-a39a-c39f505d8686" containerName="payload-pre-processing"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455595    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cff9b9b-3a83-4704-a39a-c39f505d8686" containerName="payload-pre-processing"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455739    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad206f87-2062-48a7-b2c6-df915c4a7a63" containerName="payload-processing"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455754    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455765    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebf9995a-bd67-4765-b6d8-0e1cfeced751" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455776    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455784    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455792    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455802    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="4910eacd-0342-4cd2-ad5d-fc5ead694dbb" containerName="maas-api"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455811    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1a05c6c-45d8-49bf-b6dc-4f77e40a131b" containerName="payload-pre-processing"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455823    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="d09e8458-9745-49e1-8011-187ca69a56f1" containerName="cleanup"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455833    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfed1752-5a72-44c9-ad37-86387277f9ac" containerName="maas-api"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455845    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455856    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cb04331-7309-471d-af6f-617e00bac3ce" containerName="cleanup"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455866    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cff9b9b-3a83-4704-a39a-c39f505d8686" containerName="payload-pre-processing"
Nov 02 04:04:10.455907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.455877    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="27fed099-7c7c-4417-9192-4229fac3a586" containerName="payload-processing"
Nov 02 04:04:10.460856 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.460835    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:10.485440 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.485410    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b005c7c1-638c-4ed0-bfa3-21148da14b61-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9\" (UID: \"b005c7c1-638c-4ed0-bfa3-21148da14b61\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:10.485583 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.485535    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfbck\" (UniqueName: \"kubernetes.io/projected/b005c7c1-638c-4ed0-bfa3-21148da14b61-kube-api-access-lfbck\") pod \"payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9\" (UID: \"b005c7c1-638c-4ed0-bfa3-21148da14b61\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:10.515588 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.515519    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"]
Nov 02 04:04:10.518932 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.518903    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-3f24d0\""
Nov 02 04:04:10.519106 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.518965    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-3f24d0-dockercfg-ttfzd\""
Nov 02 04:04:10.586653 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.586580    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfbck\" (UniqueName: \"kubernetes.io/projected/b005c7c1-638c-4ed0-bfa3-21148da14b61-kube-api-access-lfbck\") pod \"payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9\" (UID: \"b005c7c1-638c-4ed0-bfa3-21148da14b61\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:10.586849 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.586769    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b005c7c1-638c-4ed0-bfa3-21148da14b61-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9\" (UID: \"b005c7c1-638c-4ed0-bfa3-21148da14b61\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:10.587548 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.587524    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b005c7c1-638c-4ed0-bfa3-21148da14b61-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9\" (UID: \"b005c7c1-638c-4ed0-bfa3-21148da14b61\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:10.644612 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.644585    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfbck\" (UniqueName: \"kubernetes.io/projected/b005c7c1-638c-4ed0-bfa3-21148da14b61-kube-api-access-lfbck\") pod \"payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9\" (UID: \"b005c7c1-638c-4ed0-bfa3-21148da14b61\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:10.775046 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:10.774964    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:11.147194 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:11.147163    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"]
Nov 02 04:04:11.148035 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:04:11.148000    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb005c7c1_638c_4ed0_bfa3_21148da14b61.slice/crio-42b27c580733793e20e65f6b537a40b4dac099c82b471b7fb303083ef7d0cfab WatchSource:0}: Error finding container 42b27c580733793e20e65f6b537a40b4dac099c82b471b7fb303083ef7d0cfab: Status 404 returned error can't find the container with id 42b27c580733793e20e65f6b537a40b4dac099c82b471b7fb303083ef7d0cfab
Nov 02 04:04:11.898490 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:11.898458    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9" event={"ID":"b005c7c1-638c-4ed0-bfa3-21148da14b61","Type":"ContainerStarted","Data":"4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d"}
Nov 02 04:04:11.898701 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:11.898505    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9" event={"ID":"b005c7c1-638c-4ed0-bfa3-21148da14b61","Type":"ContainerStarted","Data":"42b27c580733793e20e65f6b537a40b4dac099c82b471b7fb303083ef7d0cfab"}
Nov 02 04:04:11.898701 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:11.898535    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:12.021658 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:12.021580    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9" podStartSLOduration=2.021567094 podStartE2EDuration="2.021567094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:04:12.017316104 +0000 UTC m=+4595.812664670" watchObservedRunningTime="2026-11-02 04:04:12.021567094 +0000 UTC m=+4595.816915658"
Nov 02 04:04:13.568900 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:13.568861    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:13.861993 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:13.861887    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:14.016197 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:14.016165    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:16.267644 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:16.267597    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:16.427174 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:16.427142    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:18.372513 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:18.372481    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"]
Nov 02 04:04:18.480088 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:18.480057    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"]
Nov 02 04:04:18.577433 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:18.577391    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:18.762432 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:18.762382    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:18.924691 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:18.924658    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:22.906969 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:22.906943    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:04:23.373693 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:23.373659    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp" podUID="32472e85-b15a-469c-9a32-efb1b361c80d" containerName="payload-processing" containerID="cri-o://750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00" gracePeriod=25
Nov 02 04:04:23.472151 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:23.472119    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:23.480444 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:23.480415    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl" podUID="bcfdd5fd-392d-49fc-b7e3-98d12700017e" containerName="payload-pre-processing" containerID="cri-o://81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00" gracePeriod=25
Nov 02 04:04:24.246403 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.246360    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" containerID="cri-o://7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef" gracePeriod=45
Nov 02 04:04:24.262197 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.262132    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main" containerID="cri-o://2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b" gracePeriod=45
Nov 02 04:04:24.545689 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.545660    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 04:04:24.563039 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.563015    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 04:04:24.677642 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677593    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-model-cache\") pod \"71c3e132-eb14-4eed-ad62-b7018519081b\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") "
Nov 02 04:04:24.677882 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677654    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-home\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-home\") pod \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") "
Nov 02 04:04:24.677882 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677681    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-tmp-dir\") pod \"71c3e132-eb14-4eed-ad62-b7018519081b\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") "
Nov 02 04:04:24.677882 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677753    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tls-certs\") pod \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") "
Nov 02 04:04:24.677882 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677795    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tmp-dir\") pod \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") "
Nov 02 04:04:24.677882 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677827    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-dshm\") pod \"71c3e132-eb14-4eed-ad62-b7018519081b\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") "
Nov 02 04:04:24.677882 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677864    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71c3e132-eb14-4eed-ad62-b7018519081b-tls-certs\" (UniqueName: \"kubernetes.io/secret/71c3e132-eb14-4eed-ad62-b7018519081b-tls-certs\") pod \"71c3e132-eb14-4eed-ad62-b7018519081b\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") "
Nov 02 04:04:24.677882 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677903    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-model-cache" pod "71c3e132-eb14-4eed-ad62-b7018519081b" (UID: "71c3e132-eb14-4eed-ad62-b7018519081b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:24.678457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677909    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-home" pod "7d7ee11e-a411-4249-ba0d-89e5bdc96760" (UID: "7d7ee11e-a411-4249-ba0d-89e5bdc96760"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:24.678457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.677941    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d7ee11e-a411-4249-ba0d-89e5bdc96760-kube-api-access-wfr9t\" (UniqueName: \"kubernetes.io/projected/7d7ee11e-a411-4249-ba0d-89e5bdc96760-kube-api-access-wfr9t\") pod \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") "
Nov 02 04:04:24.678457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678002    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-dshm\") pod \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") "
Nov 02 04:04:24.678457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678098    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-home\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-home\") pod \"71c3e132-eb14-4eed-ad62-b7018519081b\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") "
Nov 02 04:04:24.678457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678138    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-model-cache\") pod \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\" (UID: \"7d7ee11e-a411-4249-ba0d-89e5bdc96760\") "
Nov 02 04:04:24.678457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678145    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tmp-dir" pod "7d7ee11e-a411-4249-ba0d-89e5bdc96760" (UID: "7d7ee11e-a411-4249-ba0d-89e5bdc96760"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:24.678457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678201    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71c3e132-eb14-4eed-ad62-b7018519081b-kube-api-access-mpzls\" (UniqueName: \"kubernetes.io/projected/71c3e132-eb14-4eed-ad62-b7018519081b-kube-api-access-mpzls\") pod \"71c3e132-eb14-4eed-ad62-b7018519081b\" (UID: \"71c3e132-eb14-4eed-ad62-b7018519081b\") "
Nov 02 04:04:24.678859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678477    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-tmp-dir" pod "71c3e132-eb14-4eed-ad62-b7018519081b" (UID: "71c3e132-eb14-4eed-ad62-b7018519081b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:24.678859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678688    4407 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-model-cache\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.678859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678709    4407 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-home\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.678859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678726    4407 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-tmp-dir\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.678859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678738    4407 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tmp-dir\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.679125 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.678960    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-model-cache" pod "7d7ee11e-a411-4249-ba0d-89e5bdc96760" (UID: "7d7ee11e-a411-4249-ba0d-89e5bdc96760"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:24.679125 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.679113    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-home" pod "71c3e132-eb14-4eed-ad62-b7018519081b" (UID: "71c3e132-eb14-4eed-ad62-b7018519081b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:24.681262 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.681233    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-dshm" pod "7d7ee11e-a411-4249-ba0d-89e5bdc96760" (UID: "7d7ee11e-a411-4249-ba0d-89e5bdc96760"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:24.681559 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.681537    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-dshm" pod "71c3e132-eb14-4eed-ad62-b7018519081b" (UID: "71c3e132-eb14-4eed-ad62-b7018519081b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:24.681668 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.681585    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d7ee11e-a411-4249-ba0d-89e5bdc96760-kube-api-access-wfr9t" pod "7d7ee11e-a411-4249-ba0d-89e5bdc96760" (UID: "7d7ee11e-a411-4249-ba0d-89e5bdc96760"). InnerVolumeSpecName "kube-api-access-wfr9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:24.681785 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.681763    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tls-certs" pod "7d7ee11e-a411-4249-ba0d-89e5bdc96760" (UID: "7d7ee11e-a411-4249-ba0d-89e5bdc96760"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:04:24.682003 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.681987    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71c3e132-eb14-4eed-ad62-b7018519081b-kube-api-access-mpzls" pod "71c3e132-eb14-4eed-ad62-b7018519081b" (UID: "71c3e132-eb14-4eed-ad62-b7018519081b"). InnerVolumeSpecName "kube-api-access-mpzls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:24.682443 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.682427    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71c3e132-eb14-4eed-ad62-b7018519081b-tls-certs" pod "71c3e132-eb14-4eed-ad62-b7018519081b" (UID: "71c3e132-eb14-4eed-ad62-b7018519081b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:04:24.779932 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.779852    4407 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-home\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.779932 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.779883    4407 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-model-cache\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.779932 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.779897    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpzls\" (UniqueName: \"kubernetes.io/projected/71c3e132-eb14-4eed-ad62-b7018519081b-kube-api-access-mpzls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.779932 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.779912    4407 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d7ee11e-a411-4249-ba0d-89e5bdc96760-tls-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.779932 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.779924    4407 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71c3e132-eb14-4eed-ad62-b7018519081b-dshm\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.780237 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.779937    4407 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c3e132-eb14-4eed-ad62-b7018519081b-tls-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.780237 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.779949    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfr9t\" (UniqueName: \"kubernetes.io/projected/7d7ee11e-a411-4249-ba0d-89e5bdc96760-kube-api-access-wfr9t\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.780237 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.779960    4407 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d7ee11e-a411-4249-ba0d-89e5bdc96760-dshm\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:24.993326 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.993294    4407 generic.go:356] "Generic (PLEG): container finished" podID="71c3e132-eb14-4eed-ad62-b7018519081b" containerID="2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b" exitCode=0
Nov 02 04:04:24.993532 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.993336    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"
Nov 02 04:04:24.993532 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.993383    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" event={"ID":"71c3e132-eb14-4eed-ad62-b7018519081b","Type":"ContainerDied","Data":"2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b"}
Nov 02 04:04:24.993532 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.993432    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5" event={"ID":"71c3e132-eb14-4eed-ad62-b7018519081b","Type":"ContainerDied","Data":"870eafbd63cd816b596a4b132ed46e812ae46a77ec522ec65dc499a89386b619"}
Nov 02 04:04:24.993532 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.993454    4407 scope.go:122] "RemoveContainer" containerID="2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b"
Nov 02 04:04:24.994794 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.994772    4407 generic.go:356] "Generic (PLEG): container finished" podID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerID="7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef" exitCode=0
Nov 02 04:04:24.994927 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.994798    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" event={"ID":"7d7ee11e-a411-4249-ba0d-89e5bdc96760","Type":"ContainerDied","Data":"7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef"}
Nov 02 04:04:24.994927 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.994824    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"
Nov 02 04:04:24.994927 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:24.994838    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" event={"ID":"7d7ee11e-a411-4249-ba0d-89e5bdc96760","Type":"ContainerDied","Data":"d27065d9eb4f71e3fe97abfa63abe1a4bf898d069651059b4a60710ad159a9aa"}
Nov 02 04:04:25.002857 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.002840    4407 scope.go:122] "RemoveContainer" containerID="2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b"
Nov 02 04:04:25.003143 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:25.003121    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b\": container with ID starting with 2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b not found: ID does not exist" containerID="2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b"
Nov 02 04:04:25.003238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.003150    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b"} err="failed to get container status \"2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b\": rpc error: code = NotFound desc = could not find container \"2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b\": container with ID starting with 2956ca77398c9e5ac43e6fb63a566d09f2d09bfff07fddb8e4ea528b378a5b4b not found: ID does not exist"
Nov 02 04:04:25.003238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.003168    4407 scope.go:122] "RemoveContainer" containerID="7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef"
Nov 02 04:04:25.012126 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.012110    4407 scope.go:122] "RemoveContainer" containerID="7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef"
Nov 02 04:04:25.012389 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:25.012369    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef\": container with ID starting with 7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef not found: ID does not exist" containerID="7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef"
Nov 02 04:04:25.012448 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.012397    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef"} err="failed to get container status \"7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef\": rpc error: code = NotFound desc = could not find container \"7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef\": container with ID starting with 7df70478fc0e63396267b8081b2bfad5e177b21050339b748068210147fa0aef not found: ID does not exist"
Nov 02 04:04:25.160459 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.160369    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"]
Nov 02 04:04:25.165124 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.165097    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/premium-w0-1f6e39-kserve-85768b88b8-zrdz5"]
Nov 02 04:04:25.273315 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.273286    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"]
Nov 02 04:04:25.277374 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.277352    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l"]
Nov 02 04:04:25.461435 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:25.461347    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-1f6e39/distinct-2-w0-1f6e39-kserve-66b6d48b8d-ldd9l" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 04:04:26.352181 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:26.352145    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" path="/var/lib/kubelet/pods/71c3e132-eb14-4eed-ad62-b7018519081b/volumes"
Nov 02 04:04:26.352807 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:26.352577    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" path="/var/lib/kubelet/pods/7d7ee11e-a411-4249-ba0d-89e5bdc96760/volumes"
Nov 02 04:04:28.406801 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:28.406749    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:29.165666 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:29.165609    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp" podUID="32472e85-b15a-469c-9a32-efb1b361c80d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:04:29.165841 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:29.165683    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl" podUID="bcfdd5fd-392d-49fc-b7e3-98d12700017e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:04:33.270849 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:33.270815    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:35.189423 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:35.189390    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:04:38.058106 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:38.058072    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:39.165721 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:39.165672    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp" podUID="32472e85-b15a-469c-9a32-efb1b361c80d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:04:39.166145 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:39.165740    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl" podUID="bcfdd5fd-392d-49fc-b7e3-98d12700017e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:04:39.765280 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:39.765245    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:04:42.316661 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:42.316602    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:43.641346 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.641277    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 04:04:43.690254 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.690219    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32472e85-b15a-469c-9a32-efb1b361c80d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32472e85-b15a-469c-9a32-efb1b361c80d-plugins-config-volume\") pod \"32472e85-b15a-469c-9a32-efb1b361c80d\" (UID: \"32472e85-b15a-469c-9a32-efb1b361c80d\") "
Nov 02 04:04:43.690455 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.690269    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32472e85-b15a-469c-9a32-efb1b361c80d-kube-api-access-dgwlz\" (UniqueName: \"kubernetes.io/projected/32472e85-b15a-469c-9a32-efb1b361c80d-kube-api-access-dgwlz\") pod \"32472e85-b15a-469c-9a32-efb1b361c80d\" (UID: \"32472e85-b15a-469c-9a32-efb1b361c80d\") "
Nov 02 04:04:43.690673 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.690648    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32472e85-b15a-469c-9a32-efb1b361c80d-plugins-config-volume" pod "32472e85-b15a-469c-9a32-efb1b361c80d" (UID: "32472e85-b15a-469c-9a32-efb1b361c80d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:04:43.690858 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.690839    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32472e85-b15a-469c-9a32-efb1b361c80d-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:43.692976 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.692951    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32472e85-b15a-469c-9a32-efb1b361c80d-kube-api-access-dgwlz" pod "32472e85-b15a-469c-9a32-efb1b361c80d" (UID: "32472e85-b15a-469c-9a32-efb1b361c80d"). InnerVolumeSpecName "kube-api-access-dgwlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:43.738031 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.738008    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 04:04:43.792293 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.792261    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcfdd5fd-392d-49fc-b7e3-98d12700017e-kube-api-access-xfqgf\" (UniqueName: \"kubernetes.io/projected/bcfdd5fd-392d-49fc-b7e3-98d12700017e-kube-api-access-xfqgf\") pod \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\" (UID: \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\") "
Nov 02 04:04:43.792527 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.792310    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bcfdd5fd-392d-49fc-b7e3-98d12700017e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfdd5fd-392d-49fc-b7e3-98d12700017e-plugins-config-volume\") pod \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\" (UID: \"bcfdd5fd-392d-49fc-b7e3-98d12700017e\") "
Nov 02 04:04:43.792608 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.792580    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgwlz\" (UniqueName: \"kubernetes.io/projected/32472e85-b15a-469c-9a32-efb1b361c80d-kube-api-access-dgwlz\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:43.792780 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.792746    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcfdd5fd-392d-49fc-b7e3-98d12700017e-plugins-config-volume" pod "bcfdd5fd-392d-49fc-b7e3-98d12700017e" (UID: "bcfdd5fd-392d-49fc-b7e3-98d12700017e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:04:43.794751 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.794724    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcfdd5fd-392d-49fc-b7e3-98d12700017e-kube-api-access-xfqgf" pod "bcfdd5fd-392d-49fc-b7e3-98d12700017e" (UID: "bcfdd5fd-392d-49fc-b7e3-98d12700017e"). InnerVolumeSpecName "kube-api-access-xfqgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:43.843385 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.843350    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:43.893695 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.893592    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfqgf\" (UniqueName: \"kubernetes.io/projected/bcfdd5fd-392d-49fc-b7e3-98d12700017e-kube-api-access-xfqgf\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:43.893695 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:43.893647    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfdd5fd-392d-49fc-b7e3-98d12700017e-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:44.070030 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.069978    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:44.116583 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.116532    4407 generic.go:356] "Generic (PLEG): container finished" podID="32472e85-b15a-469c-9a32-efb1b361c80d" containerID="750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00" exitCode=0
Nov 02 04:04:44.116812 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.116678    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"
Nov 02 04:04:44.116812 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.116687    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp" event={"ID":"32472e85-b15a-469c-9a32-efb1b361c80d","Type":"ContainerDied","Data":"750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00"}
Nov 02 04:04:44.116812 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.116760    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp" event={"ID":"32472e85-b15a-469c-9a32-efb1b361c80d","Type":"ContainerDied","Data":"260e05b5176d201075026055e504cea3f28da969cf46a5c2c03927b8fc1b0eef"}
Nov 02 04:04:44.116812 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.116787    4407 scope.go:122] "RemoveContainer" containerID="750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00"
Nov 02 04:04:44.118892 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.118865    4407 generic.go:356] "Generic (PLEG): container finished" podID="bcfdd5fd-392d-49fc-b7e3-98d12700017e" containerID="81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00" exitCode=0
Nov 02 04:04:44.119031 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.118905    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"
Nov 02 04:04:44.119031 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.118943    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl" event={"ID":"bcfdd5fd-392d-49fc-b7e3-98d12700017e","Type":"ContainerDied","Data":"81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00"}
Nov 02 04:04:44.119031 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.119018    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl" event={"ID":"bcfdd5fd-392d-49fc-b7e3-98d12700017e","Type":"ContainerDied","Data":"0c01afcf648315b1e5348df1a45a785b1994700ff303f032b126a2e5b6775ab2"}
Nov 02 04:04:44.128380 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.128360    4407 scope.go:122] "RemoveContainer" containerID="750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00"
Nov 02 04:04:44.128715 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:44.128692    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00\": container with ID starting with 750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00 not found: ID does not exist" containerID="750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00"
Nov 02 04:04:44.128822 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.128725    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00"} err="failed to get container status \"750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00\": rpc error: code = NotFound desc = could not find container \"750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00\": container with ID starting with 750d223e2e7453e47beac621bd1e4c03ffd8699468d6611479c91b2f62b13e00 not found: ID does not exist"
Nov 02 04:04:44.128822 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.128746    4407 scope.go:122] "RemoveContainer" containerID="81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00"
Nov 02 04:04:44.137359 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.137343    4407 scope.go:122] "RemoveContainer" containerID="81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00"
Nov 02 04:04:44.137664 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:44.137600    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00\": container with ID starting with 81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00 not found: ID does not exist" containerID="81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00"
Nov 02 04:04:44.137779 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.137669    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00"} err="failed to get container status \"81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00\": rpc error: code = NotFound desc = could not find container \"81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00\": container with ID starting with 81624535c02427aa27162f7f4a6d15326882c6535aebf60979a9204e20e1fb00 not found: ID does not exist"
Nov 02 04:04:44.231385 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.231291    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:44.285471 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.285442    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"]
Nov 02 04:04:44.289188 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.289166    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-3b27a5-78dcc69455-xd7nl"]
Nov 02 04:04:44.351278 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.351249    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bcfdd5fd-392d-49fc-b7e3-98d12700017e" path="/var/lib/kubelet/pods/bcfdd5fd-392d-49fc-b7e3-98d12700017e/volumes"
Nov 02 04:04:44.402989 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.402963    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"]
Nov 02 04:04:44.409991 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.409966    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-3b27a5-79b5c57585-mgdcp"]
Nov 02 04:04:44.752218 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:44.752181    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:46.351741 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:46.351704    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32472e85-b15a-469c-9a32-efb1b361c80d" path="/var/lib/kubelet/pods/32472e85-b15a-469c-9a32-efb1b361c80d/volumes"
Nov 02 04:04:46.784266 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:46.784227    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:46.937157 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:46.937125    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:48.533756 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:48.533665    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:50.090809 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:50.090761    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:50.263658 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:50.263602    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:50.503891 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:50.503859    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:50.676888 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:50.676856    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:50.916132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:50.916046    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:51.084741 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:51.084707    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:51.249903 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:51.249876    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:51.518596 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:51.518496    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:51.722057 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:51.722022    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:51.786528 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:51.786445    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"]
Nov 02 04:04:51.786706 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:51.786688    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy" containerID="cri-o://f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e" gracePeriod=30
Nov 02 04:04:51.915439 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:51.915409    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:52.249819 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:52.249792    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:52.920569 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:52.920535    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:53.075821 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:53.075791    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:54.505893 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:54.505861    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:56.298176 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:56.298140    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:57.049064 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.049038    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 04:04:57.152596 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152501    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istiod-ca-cert\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.152596 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152548    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-socket\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.152881 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152605    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-data\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.152881 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152660    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-kube-api-access-hxgdr\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-kube-api-access-hxgdr\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.152881 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152711    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-envoy\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.152881 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152741    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-podinfo\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.152881 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152840    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-credential-socket\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.152881 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152872    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-socket" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:57.153213 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152886    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-token\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-token\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.153213 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152917    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-certs\") pod \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\" (UID: \"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3\") "
Nov 02 04:04:57.153213 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.152930    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istiod-ca-cert" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:04:57.153213 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.153115    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-credential-socket" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:57.153407 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.153251    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-certs" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:57.153407 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.153276    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-data" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:57.153407 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.153391    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.153407 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.153403    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.153832 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.153413    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.153832 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.153426    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.153832 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.153435    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.155226 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.155201    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-token" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:57.155226 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.155217    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-kube-api-access-hxgdr" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "kube-api-access-hxgdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:04:57.155377 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.155224    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-envoy" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:04:57.155377 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.155288    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-podinfo" pod "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" (UID: "a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:04:57.183809 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.183782    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"]
Nov 02 04:04:57.195759 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.195735    4407 generic.go:356] "Generic (PLEG): container finished" podID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerID="f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e" exitCode=0
Nov 02 04:04:57.195913 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.195838    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"
Nov 02 04:04:57.195913 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.195881    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" event={"ID":"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3","Type":"ContainerDied","Data":"f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e"}
Nov 02 04:04:57.196078 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.195965    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc" event={"ID":"a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3","Type":"ContainerDied","Data":"cef6cf89d70a37b920b09263fcce99118ea3da81b1470991319d0336ff5f611f"}
Nov 02 04:04:57.196078 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.195986    4407 scope.go:122] "RemoveContainer" containerID="f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e"
Nov 02 04:04:57.206274 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.206257    4407 scope.go:122] "RemoveContainer" containerID="6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd"
Nov 02 04:04:57.215468 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.215450    4407 scope.go:122] "RemoveContainer" containerID="f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e"
Nov 02 04:04:57.215784 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:57.215764    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e\": container with ID starting with f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e not found: ID does not exist" containerID="f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e"
Nov 02 04:04:57.215855 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.215793    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e"} err="failed to get container status \"f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e\": rpc error: code = NotFound desc = could not find container \"f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e\": container with ID starting with f27ced1c7601eb22aea0425a75bd818122e6a15823a43784fb87d675fd12ea2e not found: ID does not exist"
Nov 02 04:04:57.215855 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.215812    4407 scope.go:122] "RemoveContainer" containerID="6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd"
Nov 02 04:04:57.216090 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:04:57.216075    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd\": container with ID starting with 6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd not found: ID does not exist" containerID="6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd"
Nov 02 04:04:57.216139 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.216092    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd"} err="failed to get container status \"6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd\": rpc error: code = NotFound desc = could not find container \"6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd\": container with ID starting with 6174af3bb72b7db9303c7028ee7dbc4a78e588a0264787e43e3e44215b336efd not found: ID does not exist"
Nov 02 04:04:57.254535 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.254503    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxgdr\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-kube-api-access-hxgdr\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.254535 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.254530    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.254535 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.254539    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.254841 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.254548    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:04:57.334224 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.334180    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:57.335681 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.335657    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"]
Nov 02 04:04:57.341153 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.341128    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d94cc3-openshift-default-5d95d97d45-ldstc"]
Nov 02 04:04:57.715950 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.715920    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:57.877166 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:57.877138    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:58.065244 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:58.065186    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:04:58.350887 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:58.350806    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" path="/var/lib/kubelet/pods/a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3/volumes"
Nov 02 04:04:58.510401 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:04:58.510376    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:00.914475 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:00.914445    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:02.560661 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:02.560604    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:02.849372 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:02.849292    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:03.006453 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:03.006423    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:03.422636 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:03.422590    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:03.465667 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:03.465636    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"]
Nov 02 04:05:03.611761 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:03.611734    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:04.957725 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:04.957684    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:05.111238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:05.111209    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:05.188568 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:05.188534    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:05:08.456560 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:08.456528    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:09.446799 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:09.446763    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:09.624729 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:09.624678    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:09.767201 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:09.767166    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"]
Nov 02 04:05:09.767400 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:09.767380    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy" containerID="cri-o://68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3" gracePeriod=30
Nov 02 04:05:09.809876 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:09.809836    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:10.209668 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:10.209519    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:10.848444 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:10.848402    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:11.011322 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:11.011289    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:11.895492 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:11.895458    4407 patch_prober.go:29] interesting pod/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 04:05:11.895976 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:11.895527    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 04:05:12.206078 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.205982    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj" podUID="d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" containerName="main" containerID="cri-o://1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac" gracePeriod=45
Nov 02 04:05:12.471888 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.471831    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 04:05:12.560930 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.560891    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tls-certs\") pod \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") "
Nov 02 04:05:12.561108 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.560951    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-home\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-home\") pod \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") "
Nov 02 04:05:12.561108 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.560975    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-model-cache\") pod \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") "
Nov 02 04:05:12.561108 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561028    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-dshm\") pod \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") "
Nov 02 04:05:12.561268 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561139    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-kube-api-access-p6zgb\" (UniqueName: \"kubernetes.io/projected/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-kube-api-access-p6zgb\") pod \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") "
Nov 02 04:05:12.561268 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561250    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tmp-dir\") pod \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\" (UID: \"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2\") "
Nov 02 04:05:12.561372 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561254    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-home" pod "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" (UID: "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:12.561372 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561269    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-model-cache" pod "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" (UID: "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:12.561581 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561556    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tmp-dir" pod "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" (UID: "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:12.561734 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561716    4407 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tmp-dir\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:12.561803 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561739    4407 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-home\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:12.561803 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.561752    4407 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-model-cache\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:12.563445 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.563425    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-dshm" pod "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" (UID: "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:12.563525 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.563487    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-kube-api-access-p6zgb" pod "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" (UID: "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2"). InnerVolumeSpecName "kube-api-access-p6zgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:12.563525 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.563495    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tls-certs" pod "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" (UID: "d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:12.663180 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.663149    4407 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-tls-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:12.663180 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.663177    4407 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-dshm\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:12.663180 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:12.663190    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6zgb\" (UniqueName: \"kubernetes.io/projected/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2-kube-api-access-p6zgb\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:13.288898 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.288865    4407 generic.go:356] "Generic (PLEG): container finished" podID="d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" containerID="1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac" exitCode=0
Nov 02 04:05:13.289289 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.288915    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"
Nov 02 04:05:13.289289 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.288944    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj" event={"ID":"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2","Type":"ContainerDied","Data":"1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac"}
Nov 02 04:05:13.289289 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.288986    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj" event={"ID":"d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2","Type":"ContainerDied","Data":"b6f98a2db77c163712ad84820d47fabbb8cc1887d2f7b967f096772e6928e0fd"}
Nov 02 04:05:13.289289 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.289004    4407 scope.go:122] "RemoveContainer" containerID="1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac"
Nov 02 04:05:13.298890 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.298874    4407 scope.go:122] "RemoveContainer" containerID="1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac"
Nov 02 04:05:13.299149 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:13.299128    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac\": container with ID starting with 1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac not found: ID does not exist" containerID="1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac"
Nov 02 04:05:13.299202 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.299157    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac"} err="failed to get container status \"1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac\": rpc error: code = NotFound desc = could not find container \"1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac\": container with ID starting with 1fc52ae0159576f6c9d5b90e04ab1c2d3466a95e54414e6aa8660852b18bc4ac not found: ID does not exist"
Nov 02 04:05:13.454676 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.454644    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"]
Nov 02 04:05:13.505360 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.505328    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-ca6546/distinct-2-w2-ca6546-kserve-85fc48cfff-tn8dj"]
Nov 02 04:05:13.577852 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:13.577778    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:14.356567 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:14.356523    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" path="/var/lib/kubelet/pods/d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2/volumes"
Nov 02 04:05:15.042469 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.042409    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 04:05:15.157403 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.157366    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"]
Nov 02 04:05:15.168945 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.168915    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"]
Nov 02 04:05:15.191074 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191048    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istiod-ca-cert\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191100    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-socket\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191147    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-kube-api-access-t62jl\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-kube-api-access-t62jl\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191189    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-podinfo\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191235    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-certs\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191442 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191279    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-credential-socket\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191442 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191306    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-data\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191442 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191405    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-envoy\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191588 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191438    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-token\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-token\") pod \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\" (UID: \"df0fc959-11b3-40cc-ab7c-6dd2474b076d\") "
Nov 02 04:05:15.191588 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191455    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-socket" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:15.191588 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191552    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istiod-ca-cert" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:05:15.192068 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191641    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-credential-socket" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:15.192068 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.191755    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-data" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:15.192068 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.192053    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-certs" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:15.192518 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.192260    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.192518 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.192330    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.192518 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.192463    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.192518 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.192478    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.192518 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.192492    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.193550 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.193524    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-podinfo" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:05:15.193902 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.193882    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-kube-api-access-t62jl" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "kube-api-access-t62jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:15.194029 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.194014    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-envoy" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:15.194029 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.194018    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-token" pod "df0fc959-11b3-40cc-ab7c-6dd2474b076d" (UID: "df0fc959-11b3-40cc-ab7c-6dd2474b076d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:15.293190 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.293112    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.293190 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.293141    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.293190 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.293152    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t62jl\" (UniqueName: \"kubernetes.io/projected/df0fc959-11b3-40cc-ab7c-6dd2474b076d-kube-api-access-t62jl\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.293190 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.293161    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0fc959-11b3-40cc-ab7c-6dd2474b076d-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:15.302190 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.302156    4407 generic.go:356] "Generic (PLEG): container finished" podID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerID="68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3" exitCode=0
Nov 02 04:05:15.302347 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.302201    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" event={"ID":"df0fc959-11b3-40cc-ab7c-6dd2474b076d","Type":"ContainerDied","Data":"68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3"}
Nov 02 04:05:15.302347 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.302210    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"
Nov 02 04:05:15.302347 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.302243    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2" event={"ID":"df0fc959-11b3-40cc-ab7c-6dd2474b076d","Type":"ContainerDied","Data":"f77a9d078a4e2b2f02450c45ec38abb03162b9d168a34152ea1b72ffb22b432b"}
Nov 02 04:05:15.302347 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.302264    4407 scope.go:122] "RemoveContainer" containerID="68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3"
Nov 02 04:05:15.306672 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.306646    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:15.312911 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.312892    4407 scope.go:122] "RemoveContainer" containerID="5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a"
Nov 02 04:05:15.321566 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.321542    4407 scope.go:122] "RemoveContainer" containerID="68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3"
Nov 02 04:05:15.321856 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:15.321839    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3\": container with ID starting with 68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3 not found: ID does not exist" containerID="68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3"
Nov 02 04:05:15.321938 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.321864    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3"} err="failed to get container status \"68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3\": rpc error: code = NotFound desc = could not find container \"68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3\": container with ID starting with 68c920b88a729e284b5830ec2d67acbc0b1bdb2e6fe7923840444f8814d9c7b3 not found: ID does not exist"
Nov 02 04:05:15.321938 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.321881    4407 scope.go:122] "RemoveContainer" containerID="5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a"
Nov 02 04:05:15.322149 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:15.322129    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a\": container with ID starting with 5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a not found: ID does not exist" containerID="5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a"
Nov 02 04:05:15.322196 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.322156    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a"} err="failed to get container status \"5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a\": rpc error: code = NotFound desc = could not find container \"5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a\": container with ID starting with 5d7d7efdec279ddfd377de52ec0f7a2af89c476e5a4b9320269d15a11c06b66a not found: ID does not exist"
Nov 02 04:05:15.577927 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.577849    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"]
Nov 02 04:05:15.581764 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:15.581740    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-3b27a5-openshift-default-845fbbf48c-mtbc2"]
Nov 02 04:05:16.351022 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:16.350987    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" path="/var/lib/kubelet/pods/df0fc959-11b3-40cc-ab7c-6dd2474b076d/volumes"
Nov 02 04:05:18.486141 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.486097    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main" containerID="cri-o://8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d" gracePeriod=45
Nov 02 04:05:18.736332 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.736278    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 04:05:18.832329 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832290    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-dshm\") pod \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") "
Nov 02 04:05:18.832520 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832381    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tmp-dir\") pod \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") "
Nov 02 04:05:18.832520 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832423    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-kube-api-access-vwgqd\" (UniqueName: \"kubernetes.io/projected/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-kube-api-access-vwgqd\") pod \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") "
Nov 02 04:05:18.832520 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832455    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tls-certs\" (UniqueName: \"kubernetes.io/secret/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tls-certs\") pod \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") "
Nov 02 04:05:18.832520 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832480    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-home\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-home\") pod \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") "
Nov 02 04:05:18.832792 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832652    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-model-cache\") pod \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\" (UID: \"ec2f3b6e-98d1-42bc-8718-a15b56f874a9\") "
Nov 02 04:05:18.832792 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832685    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tmp-dir" pod "ec2f3b6e-98d1-42bc-8718-a15b56f874a9" (UID: "ec2f3b6e-98d1-42bc-8718-a15b56f874a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:18.832792 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832776    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-home" pod "ec2f3b6e-98d1-42bc-8718-a15b56f874a9" (UID: "ec2f3b6e-98d1-42bc-8718-a15b56f874a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:18.832935 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.832895    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-model-cache" pod "ec2f3b6e-98d1-42bc-8718-a15b56f874a9" (UID: "ec2f3b6e-98d1-42bc-8718-a15b56f874a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:18.833136 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.833118    4407 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tmp-dir\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:18.833195 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.833140    4407 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-home\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:18.833195 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.833149    4407 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-model-cache\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:18.834714 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.834697    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-dshm" pod "ec2f3b6e-98d1-42bc-8718-a15b56f874a9" (UID: "ec2f3b6e-98d1-42bc-8718-a15b56f874a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:18.835151 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.835127    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-kube-api-access-vwgqd" pod "ec2f3b6e-98d1-42bc-8718-a15b56f874a9" (UID: "ec2f3b6e-98d1-42bc-8718-a15b56f874a9"). InnerVolumeSpecName "kube-api-access-vwgqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:18.835151 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.835135    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tls-certs" pod "ec2f3b6e-98d1-42bc-8718-a15b56f874a9" (UID: "ec2f3b6e-98d1-42bc-8718-a15b56f874a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:18.934777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.934737    4407 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-dshm\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:18.934969 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.934796    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwgqd\" (UniqueName: \"kubernetes.io/projected/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-kube-api-access-vwgqd\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:18.934969 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:18.934808    4407 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec2f3b6e-98d1-42bc-8718-a15b56f874a9-tls-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:19.082047 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.082006    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:19.325519 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.325474    4407 generic.go:356] "Generic (PLEG): container finished" podID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerID="8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d" exitCode=0
Nov 02 04:05:19.325709 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.325539    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"
Nov 02 04:05:19.325709 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.325560    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" event={"ID":"ec2f3b6e-98d1-42bc-8718-a15b56f874a9","Type":"ContainerDied","Data":"8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d"}
Nov 02 04:05:19.325709 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.325604    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t" event={"ID":"ec2f3b6e-98d1-42bc-8718-a15b56f874a9","Type":"ContainerDied","Data":"f5a478464840ca071a20934a11bb7647c6c18da4df10c836611276bf7805e859"}
Nov 02 04:05:19.325709 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.325637    4407 scope.go:122] "RemoveContainer" containerID="8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d"
Nov 02 04:05:19.336002 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.335983    4407 scope.go:122] "RemoveContainer" containerID="8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d"
Nov 02 04:05:19.336275 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:19.336256    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d\": container with ID starting with 8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d not found: ID does not exist" containerID="8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d"
Nov 02 04:05:19.336324 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.336283    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d"} err="failed to get container status \"8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d\": rpc error: code = NotFound desc = could not find container \"8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d\": container with ID starting with 8d34b176b7d246cd49bb3fe2e98c3a7a3150484b06a0ab0e4b078db61d3ba13d not found: ID does not exist"
Nov 02 04:05:19.491258 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.491227    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"]
Nov 02 04:05:19.496217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:19.496194    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-398828/premium-w4-398828-kserve-574875d9d4-wrm5t"]
Nov 02 04:05:20.350598 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:20.350567    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" path="/var/lib/kubelet/pods/ec2f3b6e-98d1-42bc-8718-a15b56f874a9/volumes"
Nov 02 04:05:21.251864 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:21.251829    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:05:22.633729 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:22.633698    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:22.796541 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:22.796508    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:23.151315 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.151279    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"]
Nov 02 04:05:23.151519 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.151496    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265" podUID="90cc541b-2f24-4f28-8045-99e7736a1b7f" containerName="maas-api" containerID="cri-o://91e20546e94f1a148f9d61f00d1fe2cc78f2dabcb62ee5b2c73bace4f1f06ee5" gracePeriod=30
Nov 02 04:05:23.263283 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.263252    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:23.351850 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.351821    4407 generic.go:356] "Generic (PLEG): container finished" podID="90cc541b-2f24-4f28-8045-99e7736a1b7f" containerID="91e20546e94f1a148f9d61f00d1fe2cc78f2dabcb62ee5b2c73bace4f1f06ee5" exitCode=0
Nov 02 04:05:23.352010 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.351898    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265" event={"ID":"90cc541b-2f24-4f28-8045-99e7736a1b7f","Type":"ContainerDied","Data":"91e20546e94f1a148f9d61f00d1fe2cc78f2dabcb62ee5b2c73bace4f1f06ee5"}
Nov 02 04:05:23.419135 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.419075    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 04:05:23.589476 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.589439    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90cc541b-2f24-4f28-8045-99e7736a1b7f-kube-api-access-grhns\" (UniqueName: \"kubernetes.io/projected/90cc541b-2f24-4f28-8045-99e7736a1b7f-kube-api-access-grhns\") pod \"90cc541b-2f24-4f28-8045-99e7736a1b7f\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") "
Nov 02 04:05:23.589699 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.589552    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90cc541b-2f24-4f28-8045-99e7736a1b7f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90cc541b-2f24-4f28-8045-99e7736a1b7f-openshift-service-ca\") pod \"90cc541b-2f24-4f28-8045-99e7736a1b7f\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") "
Nov 02 04:05:23.589699 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.589606    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-metrics-certs\") pod \"90cc541b-2f24-4f28-8045-99e7736a1b7f\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") "
Nov 02 04:05:23.589699 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.589685    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls\") pod \"90cc541b-2f24-4f28-8045-99e7736a1b7f\" (UID: \"90cc541b-2f24-4f28-8045-99e7736a1b7f\") "
Nov 02 04:05:23.589986 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.589926    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90cc541b-2f24-4f28-8045-99e7736a1b7f-openshift-service-ca" pod "90cc541b-2f24-4f28-8045-99e7736a1b7f" (UID: "90cc541b-2f24-4f28-8045-99e7736a1b7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:05:23.590149 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.590134    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90cc541b-2f24-4f28-8045-99e7736a1b7f-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:23.591958 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.591935    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-metrics-certs" pod "90cc541b-2f24-4f28-8045-99e7736a1b7f" (UID: "90cc541b-2f24-4f28-8045-99e7736a1b7f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:23.592194 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.592162    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls" pod "90cc541b-2f24-4f28-8045-99e7736a1b7f" (UID: "90cc541b-2f24-4f28-8045-99e7736a1b7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:23.592287 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.592230    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90cc541b-2f24-4f28-8045-99e7736a1b7f-kube-api-access-grhns" pod "90cc541b-2f24-4f28-8045-99e7736a1b7f" (UID: "90cc541b-2f24-4f28-8045-99e7736a1b7f"). InnerVolumeSpecName "kube-api-access-grhns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:23.691239 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.691133    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:23.691239 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.691172    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90cc541b-2f24-4f28-8045-99e7736a1b7f-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:23.691239 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:23.691187    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grhns\" (UniqueName: \"kubernetes.io/projected/90cc541b-2f24-4f28-8045-99e7736a1b7f-kube-api-access-grhns\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:24.365643 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:24.365588    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265" event={"ID":"90cc541b-2f24-4f28-8045-99e7736a1b7f","Type":"ContainerDied","Data":"2680393414acf69bd3f025cc24b4f9ff77d3beac920cb25a359a72ff99891d8b"}
Nov 02 04:05:24.365870 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:24.365720    4407 scope.go:122] "RemoveContainer" containerID="91e20546e94f1a148f9d61f00d1fe2cc78f2dabcb62ee5b2c73bace4f1f06ee5"
Nov 02 04:05:24.365870 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:24.365750    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"
Nov 02 04:05:24.532089 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:24.532055    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"]
Nov 02 04:05:24.582905 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:24.582872    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-1f6e39-7c9ff79d44-nj265"]
Nov 02 04:05:24.855841 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:24.855806    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:25.055756 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:25.055722    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:25.304842 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:25.304812    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:25.779341 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:25.779287    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:26.351703 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:26.351670    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90cc541b-2f24-4f28-8045-99e7736a1b7f" path="/var/lib/kubelet/pods/90cc541b-2f24-4f28-8045-99e7736a1b7f/volumes"
Nov 02 04:05:30.178726 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.178652    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main" containerID="cri-o://c66590ffe9c18b056e7045184e89ea00f73f399e0b5db0f56f7f27ccf05dbc9b" gracePeriod=45
Nov 02 04:05:30.189034 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.189008    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main" containerID="cri-o://38485bfaf6833208b3f552d2d8f3a32c7b8a4ce0e87e4bee5c45e9cc38007c74" gracePeriod=45
Nov 02 04:05:30.326156 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.326120    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:30.403294 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.403265    4407 generic.go:356] "Generic (PLEG): container finished" podID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerID="38485bfaf6833208b3f552d2d8f3a32c7b8a4ce0e87e4bee5c45e9cc38007c74" exitCode=0
Nov 02 04:05:30.403465 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.403411    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" event={"ID":"f44a93a8-19e2-47e1-969f-c156f5c97772","Type":"ContainerDied","Data":"38485bfaf6833208b3f552d2d8f3a32c7b8a4ce0e87e4bee5c45e9cc38007c74"}
Nov 02 04:05:30.405265 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.405237    4407 generic.go:356] "Generic (PLEG): container finished" podID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerID="c66590ffe9c18b056e7045184e89ea00f73f399e0b5db0f56f7f27ccf05dbc9b" exitCode=0
Nov 02 04:05:30.405392 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.405270    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" event={"ID":"173c0706-37a9-4e23-823d-ffbf6e67be49","Type":"ContainerDied","Data":"c66590ffe9c18b056e7045184e89ea00f73f399e0b5db0f56f7f27ccf05dbc9b"}
Nov 02 04:05:30.467474 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.467449    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 04:05:30.490928 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.490905    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 04:05:30.564938 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.564891    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-dshm\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-dshm\") pod \"173c0706-37a9-4e23-823d-ffbf6e67be49\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") "
Nov 02 04:05:30.565109 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.564964    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/173c0706-37a9-4e23-823d-ffbf6e67be49-tls-certs\" (UniqueName: \"kubernetes.io/secret/173c0706-37a9-4e23-823d-ffbf6e67be49-tls-certs\") pod \"173c0706-37a9-4e23-823d-ffbf6e67be49\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") "
Nov 02 04:05:30.565109 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.564997    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-home\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-home\") pod \"f44a93a8-19e2-47e1-969f-c156f5c97772\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") "
Nov 02 04:05:30.565109 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565049    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f44a93a8-19e2-47e1-969f-c156f5c97772-kube-api-access-ccjvr\" (UniqueName: \"kubernetes.io/projected/f44a93a8-19e2-47e1-969f-c156f5c97772-kube-api-access-ccjvr\") pod \"f44a93a8-19e2-47e1-969f-c156f5c97772\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") "
Nov 02 04:05:30.565256 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565109    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f44a93a8-19e2-47e1-969f-c156f5c97772-tls-certs\" (UniqueName: \"kubernetes.io/secret/f44a93a8-19e2-47e1-969f-c156f5c97772-tls-certs\") pod \"f44a93a8-19e2-47e1-969f-c156f5c97772\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") "
Nov 02 04:05:30.565256 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565135    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-dshm\") pod \"f44a93a8-19e2-47e1-969f-c156f5c97772\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") "
Nov 02 04:05:30.565256 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565166    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-tmp-dir\") pod \"f44a93a8-19e2-47e1-969f-c156f5c97772\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") "
Nov 02 04:05:30.565256 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565250    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-model-cache\") pod \"f44a93a8-19e2-47e1-969f-c156f5c97772\" (UID: \"f44a93a8-19e2-47e1-969f-c156f5c97772\") "
Nov 02 04:05:30.565480 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565295    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-model-cache\") pod \"173c0706-37a9-4e23-823d-ffbf6e67be49\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") "
Nov 02 04:05:30.565480 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565350    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-tmp-dir\") pod \"173c0706-37a9-4e23-823d-ffbf6e67be49\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") "
Nov 02 04:05:30.565480 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565430    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/173c0706-37a9-4e23-823d-ffbf6e67be49-kube-api-access-h4q5s\" (UniqueName: \"kubernetes.io/projected/173c0706-37a9-4e23-823d-ffbf6e67be49-kube-api-access-h4q5s\") pod \"173c0706-37a9-4e23-823d-ffbf6e67be49\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") "
Nov 02 04:05:30.565480 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565464    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-home\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-home\") pod \"173c0706-37a9-4e23-823d-ffbf6e67be49\" (UID: \"173c0706-37a9-4e23-823d-ffbf6e67be49\") "
Nov 02 04:05:30.565770 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565586    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-home" pod "f44a93a8-19e2-47e1-969f-c156f5c97772" (UID: "f44a93a8-19e2-47e1-969f-c156f5c97772"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:30.565770 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565655    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-model-cache" pod "f44a93a8-19e2-47e1-969f-c156f5c97772" (UID: "f44a93a8-19e2-47e1-969f-c156f5c97772"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:30.565898 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.565823    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-tmp-dir" pod "173c0706-37a9-4e23-823d-ffbf6e67be49" (UID: "173c0706-37a9-4e23-823d-ffbf6e67be49"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:30.566103 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.566035    4407 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-model-cache\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.566103 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.566063    4407 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-tmp-dir\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.566103 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.566080    4407 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-home\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.566352 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.566102    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-home" pod "173c0706-37a9-4e23-823d-ffbf6e67be49" (UID: "173c0706-37a9-4e23-823d-ffbf6e67be49"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:30.566352 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.566160    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-model-cache" pod "173c0706-37a9-4e23-823d-ffbf6e67be49" (UID: "173c0706-37a9-4e23-823d-ffbf6e67be49"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:30.566352 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.566209    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-tmp-dir" pod "f44a93a8-19e2-47e1-969f-c156f5c97772" (UID: "f44a93a8-19e2-47e1-969f-c156f5c97772"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:30.568197 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.568171    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173c0706-37a9-4e23-823d-ffbf6e67be49-tls-certs" pod "173c0706-37a9-4e23-823d-ffbf6e67be49" (UID: "173c0706-37a9-4e23-823d-ffbf6e67be49"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:30.568304 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.568229    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f44a93a8-19e2-47e1-969f-c156f5c97772-kube-api-access-ccjvr" pod "f44a93a8-19e2-47e1-969f-c156f5c97772" (UID: "f44a93a8-19e2-47e1-969f-c156f5c97772"). InnerVolumeSpecName "kube-api-access-ccjvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:30.568385 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.568365    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-dshm" pod "173c0706-37a9-4e23-823d-ffbf6e67be49" (UID: "173c0706-37a9-4e23-823d-ffbf6e67be49"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:30.568461 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.568438    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f44a93a8-19e2-47e1-969f-c156f5c97772-tls-certs" pod "f44a93a8-19e2-47e1-969f-c156f5c97772" (UID: "f44a93a8-19e2-47e1-969f-c156f5c97772"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:30.568531 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.568509    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173c0706-37a9-4e23-823d-ffbf6e67be49-kube-api-access-h4q5s" pod "173c0706-37a9-4e23-823d-ffbf6e67be49" (UID: "173c0706-37a9-4e23-823d-ffbf6e67be49"). InnerVolumeSpecName "kube-api-access-h4q5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:30.568730 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.568708    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-dshm" pod "f44a93a8-19e2-47e1-969f-c156f5c97772" (UID: "f44a93a8-19e2-47e1-969f-c156f5c97772"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:30.666747 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666715    4407 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44a93a8-19e2-47e1-969f-c156f5c97772-tls-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.666747 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666744    4407 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-dshm\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.666747 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666754    4407 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44a93a8-19e2-47e1-969f-c156f5c97772-tmp-dir\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.666980 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666762    4407 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-model-cache\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.666980 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666773    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4q5s\" (UniqueName: \"kubernetes.io/projected/173c0706-37a9-4e23-823d-ffbf6e67be49-kube-api-access-h4q5s\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.666980 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666782    4407 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-home\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.666980 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666792    4407 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/173c0706-37a9-4e23-823d-ffbf6e67be49-dshm\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.666980 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666804    4407 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/173c0706-37a9-4e23-823d-ffbf6e67be49-tls-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:30.666980 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:30.666812    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccjvr\" (UniqueName: \"kubernetes.io/projected/f44a93a8-19e2-47e1-969f-c156f5c97772-kube-api-access-ccjvr\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:31.411694 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.411660    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"
Nov 02 04:05:31.412132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.411665    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf" event={"ID":"173c0706-37a9-4e23-823d-ffbf6e67be49","Type":"ContainerDied","Data":"780faa6bd2fca7b3791657e9c43013f8337ea56daf220edd53b83803369091c1"}
Nov 02 04:05:31.412132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.411805    4407 scope.go:122] "RemoveContainer" containerID="c66590ffe9c18b056e7045184e89ea00f73f399e0b5db0f56f7f27ccf05dbc9b"
Nov 02 04:05:31.413148 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.412969    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"
Nov 02 04:05:31.413148 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.413000    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s" event={"ID":"f44a93a8-19e2-47e1-969f-c156f5c97772","Type":"ContainerDied","Data":"97b354e485c351428d557c102955478446b9085ffb15cdbf0bf77f4ec85016f6"}
Nov 02 04:05:31.421966 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.421952    4407 scope.go:122] "RemoveContainer" containerID="38485bfaf6833208b3f552d2d8f3a32c7b8a4ce0e87e4bee5c45e9cc38007c74"
Nov 02 04:05:31.578014 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.577982    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"]
Nov 02 04:05:31.629066 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.629033    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/simulator-w1-5ac93f-kserve-57cb964469-xsv4s"]
Nov 02 04:05:31.739301 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.739274    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"]
Nov 02 04:05:31.744836 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.744807    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-5ac93f/premium-w1-5ac93f-kserve-74bf649df6-gjwrf"]
Nov 02 04:05:31.899862 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:31.899785    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:32.071758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:32.071729    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:32.170654 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:32.170550    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"]
Nov 02 04:05:32.170877 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:32.170851    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy" containerID="cri-o://74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8" gracePeriod=30
Nov 02 04:05:32.251659 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:32.251618    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:32.351068 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:32.351037    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" path="/var/lib/kubelet/pods/173c0706-37a9-4e23-823d-ffbf6e67be49/volumes"
Nov 02 04:05:32.351483 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:32.351470    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" path="/var/lib/kubelet/pods/f44a93a8-19e2-47e1-969f-c156f5c97772/volumes"
Nov 02 04:05:33.098565 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:33.098533    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:33.983888 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:33.983858    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:34.190103 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.190068    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:34.378602 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.378574    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:34.769448 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769418    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"]
Nov 02 04:05:34.769902 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769888    4407 cpu_manager.go:414] "containerMap: removing container" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy"
Nov 02 04:05:34.769952 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769904    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy"
Nov 02 04:05:34.769952 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769916    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main"
Nov 02 04:05:34.769952 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769921    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main"
Nov 02 04:05:34.769952 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769927    4407 cpu_manager.go:414] "containerMap: removing container" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy"
Nov 02 04:05:34.769952 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769932    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy"
Nov 02 04:05:34.769952 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769942    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy"
Nov 02 04:05:34.769952 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769947    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy"
Nov 02 04:05:34.769952 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769954    4407 cpu_manager.go:414] "containerMap: removing container" podUID="32472e85-b15a-469c-9a32-efb1b361c80d" containerName="payload-processing"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769959    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32472e85-b15a-469c-9a32-efb1b361c80d" containerName="payload-processing"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769967    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bcfdd5fd-392d-49fc-b7e3-98d12700017e" containerName="payload-pre-processing"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769972    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcfdd5fd-392d-49fc-b7e3-98d12700017e" containerName="payload-pre-processing"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769979    4407 cpu_manager.go:414] "containerMap: removing container" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.769983    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770005    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770010    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770019    4407 cpu_manager.go:414] "containerMap: removing container" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770023    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770029    4407 cpu_manager.go:414] "containerMap: removing container" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770034    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60105b4e-1c8b-4fbd-b501-5608e1392a55" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770043    4407 cpu_manager.go:414] "containerMap: removing container" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770047    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770056    4407 cpu_manager.go:414] "containerMap: removing container" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770061    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770073    4407 cpu_manager.go:414] "containerMap: removing container" podUID="90cc541b-2f24-4f28-8045-99e7736a1b7f" containerName="maas-api"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770077    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90cc541b-2f24-4f28-8045-99e7736a1b7f" containerName="maas-api"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770084    4407 cpu_manager.go:414] "containerMap: removing container" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770088    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770096    4407 cpu_manager.go:414] "containerMap: removing container" podUID="d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770100    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770108    4407 cpu_manager.go:414] "containerMap: removing container" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770113    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770174    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770180    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="32472e85-b15a-469c-9a32-efb1b361c80d" containerName="payload-processing"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770186    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcfdd5fd-392d-49fc-b7e3-98d12700017e" containerName="payload-pre-processing"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770191    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="f44a93a8-19e2-47e1-969f-c156f5c97772" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770197    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8f3f2f6-d2ac-41f8-a180-39f32be5bcd3" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770202    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770207    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfb3623e-4997-4231-9052-54b65d6441f1" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770212    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ac08935-e676-4ffc-ab77-4abc80198b6c" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770218    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="71c3e132-eb14-4eed-ad62-b7018519081b" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770225    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="90cc541b-2f24-4f28-8045-99e7736a1b7f" containerName="maas-api"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770230    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec2f3b6e-98d1-42bc-8718-a15b56f874a9" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770235    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5fdcac2-0361-45f6-a8e3-a0b7368a9ac2" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770240    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="173c0706-37a9-4e23-823d-ffbf6e67be49" containerName="main"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770246    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="df0fc959-11b3-40cc-ab7c-6dd2474b076d" containerName="istio-proxy"
Nov 02 04:05:34.770682 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.770252    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d7ee11e-a411-4249-ba0d-89e5bdc96760" containerName="main"
Nov 02 04:05:34.775236 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.775214    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.821956 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.821915    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-certs\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.822404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.822374    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-data\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.822545 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.822463    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-envoy\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.822545 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.822518    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-podinfo\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.822723 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.822553    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-token\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.822723 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.822588    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-socket\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.822723 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.822688    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czjw\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-kube-api-access-5czjw\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.822902 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.822733    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdf2cf02-113e-4472-a59d-30abda30c9d3-istiod-ca-cert\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.822902 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.822756    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-credential-socket\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.869137 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.869105    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-12c92e-openshift-default-dockercfg-j6bk6\""
Nov 02 04:05:34.871403 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.871378    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"]
Nov 02 04:05:34.905464 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.905433    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:34.924076 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.923975    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-podinfo\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924076 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924026    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-token\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924076 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924059    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-socket\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924395 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924108    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czjw\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-kube-api-access-5czjw\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924395 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924142    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdf2cf02-113e-4472-a59d-30abda30c9d3-istiod-ca-cert\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924395 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924249    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-credential-socket\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924395 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924319    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-certs\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924395 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924364    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-data\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924692 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924440    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-envoy\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924692 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924553    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-socket\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924692 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924684    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-credential-socket\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.924953 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924933    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-certs\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.925016 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.924988    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-data\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.925073 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.925055    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdf2cf02-113e-4472-a59d-30abda30c9d3-istiod-ca-cert\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.927530 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.927483    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-envoy\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.927707 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.927559    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-podinfo\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.982353 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.982319    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-token\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:34.982491 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:34.982334    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czjw\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-kube-api-access-5czjw\") pod \"e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") " pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:35.090467 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:35.090377    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:35.192266 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:35.192223    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:05:35.270807 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:35.270773    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:35.516799 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:35.516773    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"]
Nov 02 04:05:35.517606 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:05:35.517579    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf2cf02_113e_4472_a59d_30abda30c9d3.slice/crio-75d6be05ac6974e6394a55a5846ccb1b79f0e608ad594e5050ba0347cbc454a5 WatchSource:0}: Error finding container 75d6be05ac6974e6394a55a5846ccb1b79f0e608ad594e5050ba0347cbc454a5: Status 404 returned error can't find the container with id 75d6be05ac6974e6394a55a5846ccb1b79f0e608ad594e5050ba0347cbc454a5
Nov 02 04:05:35.519861 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:35.519828    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:05:35.519940 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:35.519894    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:05:35.519940 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:35.519923    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:05:35.636731 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:35.636703    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:35.811181 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:35.811104    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:36.447584 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:36.447541    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" event={"ID":"bdf2cf02-113e-4472-a59d-30abda30c9d3","Type":"ContainerStarted","Data":"4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c"}
Nov 02 04:05:36.448144 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:36.447667    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" event={"ID":"bdf2cf02-113e-4472-a59d-30abda30c9d3","Type":"ContainerStarted","Data":"75d6be05ac6974e6394a55a5846ccb1b79f0e608ad594e5050ba0347cbc454a5"}
Nov 02 04:05:36.619689 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:36.619635    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" podStartSLOduration=2.619601728 podStartE2EDuration="2.619601728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:05:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:05:36.616182214 +0000 UTC m=+4680.411530772" watchObservedRunningTime="2026-11-02 04:05:36.619601728 +0000 UTC m=+4680.414950286"
Nov 02 04:05:37.091116 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.091086    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:37.096708 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.096682    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:37.425603 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.425580    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 04:05:37.455240 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.455215    4407 generic.go:356] "Generic (PLEG): container finished" podID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerID="74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8" exitCode=0
Nov 02 04:05:37.455578 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.455262    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"
Nov 02 04:05:37.455578 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.455299    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" event={"ID":"e0573a2c-b964-43a7-aa5e-8d7f9baed082","Type":"ContainerDied","Data":"74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8"}
Nov 02 04:05:37.455578 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.455345    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89" event={"ID":"e0573a2c-b964-43a7-aa5e-8d7f9baed082","Type":"ContainerDied","Data":"8cca2a7071c6fd68fa642fecfbda69a119b1f7eccc7898e662c0a475bd774229"}
Nov 02 04:05:37.455578 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.455375    4407 scope.go:122] "RemoveContainer" containerID="74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8"
Nov 02 04:05:37.455578 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.455542    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:37.456739 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.456723    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:05:37.464413 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.464391    4407 scope.go:122] "RemoveContainer" containerID="74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8"
Nov 02 04:05:37.464686 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:37.464661    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8\": container with ID starting with 74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8 not found: ID does not exist" containerID="74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8"
Nov 02 04:05:37.464797 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.464691    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8"} err="failed to get container status \"74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8\": rpc error: code = NotFound desc = could not find container \"74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8\": container with ID starting with 74c30abf174c8538f3eea3c7ba41d973127e1f5bf59262fcbaad1a5cd3dd8ef8 not found: ID does not exist"
Nov 02 04:05:37.559356 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559327    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-kube-api-access-cx7pg\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-kube-api-access-cx7pg\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.559526 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559423    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-podinfo\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.559526 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559447    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-certs\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.559526 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559486    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-data\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.559758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559534    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-envoy\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.559758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559596    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istiod-ca-cert\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.559758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559635    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-token\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-token\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.559758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559685    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-credential-socket\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.559758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559724    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-socket\") pod \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\" (UID: \"e0573a2c-b964-43a7-aa5e-8d7f9baed082\") "
Nov 02 04:05:37.560033 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559793    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-certs" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:37.560033 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.559824    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-data" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:37.560149 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.560126    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istiod-ca-cert" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:05:37.560398 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.560368    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-credential-socket" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:37.560573 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.560552    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-socket" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:37.560673 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.560574    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.560934 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.560915    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.562205 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.562181    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-token" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:37.562395 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.562376    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-envoy" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:37.562718 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.562694    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-podinfo" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:05:37.563181 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.563156    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-kube-api-access-cx7pg" pod "e0573a2c-b964-43a7-aa5e-8d7f9baed082" (UID: "e0573a2c-b964-43a7-aa5e-8d7f9baed082"). InnerVolumeSpecName "kube-api-access-cx7pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:37.662217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.662127    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.662217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.662167    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.662217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.662178    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.662217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.662187    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.662217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.662197    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx7pg\" (UniqueName: \"kubernetes.io/projected/e0573a2c-b964-43a7-aa5e-8d7f9baed082-kube-api-access-cx7pg\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.662217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.662208    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.662217 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.662216    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0573a2c-b964-43a7-aa5e-8d7f9baed082-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:37.877043 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.877009    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"]
Nov 02 04:05:37.880791 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:37.880765    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-1f6e39-openshift-default-55b56b9965-r9q89"]
Nov 02 04:05:38.350225 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:38.350191    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" path="/var/lib/kubelet/pods/e0573a2c-b964-43a7-aa5e-8d7f9baed082/volumes"
Nov 02 04:05:39.346684 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:39.346650    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:39.531728 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:39.531694    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:39.734515 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:39.734440    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:40.386634 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.386590    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"]
Nov 02 04:05:40.387079 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.386803    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t" podUID="325388df-f4cd-4944-a901-39e1da3558cb" containerName="maas-api" containerID="cri-o://e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2" gracePeriod=30
Nov 02 04:05:40.532412 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.532380    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:40.639997 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.639942    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 04:05:40.693910 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.693875    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/325388df-f4cd-4944-a901-39e1da3558cb-kube-api-access-5dwm8\" (UniqueName: \"kubernetes.io/projected/325388df-f4cd-4944-a901-39e1da3558cb-kube-api-access-5dwm8\") pod \"325388df-f4cd-4944-a901-39e1da3558cb\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") "
Nov 02 04:05:40.694097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.693941    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/325388df-f4cd-4944-a901-39e1da3558cb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/325388df-f4cd-4944-a901-39e1da3558cb-openshift-service-ca\") pod \"325388df-f4cd-4944-a901-39e1da3558cb\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") "
Nov 02 04:05:40.694097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.694033    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls\") pod \"325388df-f4cd-4944-a901-39e1da3558cb\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") "
Nov 02 04:05:40.694243 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.694096    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-metrics-certs\") pod \"325388df-f4cd-4944-a901-39e1da3558cb\" (UID: \"325388df-f4cd-4944-a901-39e1da3558cb\") "
Nov 02 04:05:40.695344 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.695300    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/325388df-f4cd-4944-a901-39e1da3558cb-openshift-service-ca" pod "325388df-f4cd-4944-a901-39e1da3558cb" (UID: "325388df-f4cd-4944-a901-39e1da3558cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:05:40.697438 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.697408    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/325388df-f4cd-4944-a901-39e1da3558cb-kube-api-access-5dwm8" pod "325388df-f4cd-4944-a901-39e1da3558cb" (UID: "325388df-f4cd-4944-a901-39e1da3558cb"). InnerVolumeSpecName "kube-api-access-5dwm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:40.697812 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.697791    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls" pod "325388df-f4cd-4944-a901-39e1da3558cb" (UID: "325388df-f4cd-4944-a901-39e1da3558cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:40.698961 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.698936    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-metrics-certs" pod "325388df-f4cd-4944-a901-39e1da3558cb" (UID: "325388df-f4cd-4944-a901-39e1da3558cb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:40.796050 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.796010    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dwm8\" (UniqueName: \"kubernetes.io/projected/325388df-f4cd-4944-a901-39e1da3558cb-kube-api-access-5dwm8\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:40.796050 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.796050    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/325388df-f4cd-4944-a901-39e1da3558cb-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:40.796262 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.796064    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:40.796262 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:40.796076    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/325388df-f4cd-4944-a901-39e1da3558cb-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:41.241182 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.241143    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:41.467672 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.467612    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:41.479545 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.479520    4407 generic.go:356] "Generic (PLEG): container finished" podID="325388df-f4cd-4944-a901-39e1da3558cb" containerID="e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2" exitCode=0
Nov 02 04:05:41.479710 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.479570    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"
Nov 02 04:05:41.479710 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.479602    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t" event={"ID":"325388df-f4cd-4944-a901-39e1da3558cb","Type":"ContainerDied","Data":"e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2"}
Nov 02 04:05:41.479710 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.479681    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t" event={"ID":"325388df-f4cd-4944-a901-39e1da3558cb","Type":"ContainerDied","Data":"5a7b5dc82fa5034b358682acefefa65b46e34002f916990238083d7c4235be8f"}
Nov 02 04:05:41.479876 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.479711    4407 scope.go:122] "RemoveContainer" containerID="e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2"
Nov 02 04:05:41.491517 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.491491    4407 scope.go:122] "RemoveContainer" containerID="e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2"
Nov 02 04:05:41.491836 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:41.491815    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2\": container with ID starting with e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2 not found: ID does not exist" containerID="e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2"
Nov 02 04:05:41.491921 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.491842    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2"} err="failed to get container status \"e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2\": rpc error: code = NotFound desc = could not find container \"e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2\": container with ID starting with e0effa7461a7ce41a59df3447c7a8b6265f41038a5abbf899e3088479668aad2 not found: ID does not exist"
Nov 02 04:05:41.646544 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.646500    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"]
Nov 02 04:05:41.697229 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:41.697188    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ca6546-86cb88b49f-5ps5t"]
Nov 02 04:05:42.323524 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:42.323488    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:42.350872 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:42.350838    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="325388df-f4cd-4944-a901-39e1da3558cb" path="/var/lib/kubelet/pods/325388df-f4cd-4944-a901-39e1da3558cb/volumes"
Nov 02 04:05:42.496636 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:42.496601    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:42.669344 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:42.669264    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:42.821041 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:42.821011    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:43.017779 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.017744    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"]
Nov 02 04:05:43.018231 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.018217    4407 cpu_manager.go:414] "containerMap: removing container" podUID="325388df-f4cd-4944-a901-39e1da3558cb" containerName="maas-api"
Nov 02 04:05:43.018281 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.018233    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="325388df-f4cd-4944-a901-39e1da3558cb" containerName="maas-api"
Nov 02 04:05:43.018281 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.018248    4407 cpu_manager.go:414] "containerMap: removing container" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy"
Nov 02 04:05:43.018281 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.018253    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy"
Nov 02 04:05:43.018376 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.018316    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="325388df-f4cd-4944-a901-39e1da3558cb" containerName="maas-api"
Nov 02 04:05:43.018376 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.018327    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0573a2c-b964-43a7-aa5e-8d7f9baed082" containerName="istio-proxy"
Nov 02 04:05:43.022945 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.022928    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:43.039068 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.039042    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:43.083306 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.083275    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-12c92e\""
Nov 02 04:05:43.083479 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.083382    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-12c92e-dockercfg-clxl7\""
Nov 02 04:05:43.122649 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.121880    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f37870b-0726-49e6-92ea-a55914f57f2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx\" (UID: \"0f37870b-0726-49e6-92ea-a55914f57f2b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:43.122649 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.121966    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj56t\" (UniqueName: \"kubernetes.io/projected/0f37870b-0726-49e6-92ea-a55914f57f2b-kube-api-access-kj56t\") pod \"payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx\" (UID: \"0f37870b-0726-49e6-92ea-a55914f57f2b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:43.137196 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.137159    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"]
Nov 02 04:05:43.168988 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.168959    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"]
Nov 02 04:05:43.172807 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.172789    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:43.220817 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.220768    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:43.223044 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.223020    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f37870b-0726-49e6-92ea-a55914f57f2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx\" (UID: \"0f37870b-0726-49e6-92ea-a55914f57f2b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:43.223174 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.223082    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj56t\" (UniqueName: \"kubernetes.io/projected/0f37870b-0726-49e6-92ea-a55914f57f2b-kube-api-access-kj56t\") pod \"payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx\" (UID: \"0f37870b-0726-49e6-92ea-a55914f57f2b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:43.223760 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.223743    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f37870b-0726-49e6-92ea-a55914f57f2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx\" (UID: \"0f37870b-0726-49e6-92ea-a55914f57f2b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:43.229734 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.229705    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"]
Nov 02 04:05:43.281404 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.281332    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj56t\" (UniqueName: \"kubernetes.io/projected/0f37870b-0726-49e6-92ea-a55914f57f2b-kube-api-access-kj56t\") pod \"payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx\" (UID: \"0f37870b-0726-49e6-92ea-a55914f57f2b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:43.324717 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.324675    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8gp4\" (UniqueName: \"kubernetes.io/projected/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-kube-api-access-v8gp4\") pod \"payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt\" (UID: \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:43.324918 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.324849    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt\" (UID: \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:43.335864 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.335840    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:43.425927 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.425893    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8gp4\" (UniqueName: \"kubernetes.io/projected/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-kube-api-access-v8gp4\") pod \"payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt\" (UID: \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:43.426104 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.426018    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt\" (UID: \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:43.426993 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.426969    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt\" (UID: \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:43.484183 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.484150    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8gp4\" (UniqueName: \"kubernetes.io/projected/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-kube-api-access-v8gp4\") pod \"payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt\" (UID: \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:43.486007 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.485988    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:43.705575 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.705544    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"]
Nov 02 04:05:43.707085 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:05:43.707060    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f37870b_0726_49e6_92ea_a55914f57f2b.slice/crio-3aa7af85b841c1624cff69829e824e960c5c7751e04b61acec4f5d556f52d5af WatchSource:0}: Error finding container 3aa7af85b841c1624cff69829e824e960c5c7751e04b61acec4f5d556f52d5af: Status 404 returned error can't find the container with id 3aa7af85b841c1624cff69829e824e960c5c7751e04b61acec4f5d556f52d5af
Nov 02 04:05:43.860962 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:05:43.860926    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7029ba4f_93aa_4d8c_aba4_b1fdaf275526.slice/crio-7407ae2e26aa0232fe64b15c3930ffe8a0b559a5b45239fb91756b451553b049 WatchSource:0}: Error finding container 7407ae2e26aa0232fe64b15c3930ffe8a0b559a5b45239fb91756b451553b049: Status 404 returned error can't find the container with id 7407ae2e26aa0232fe64b15c3930ffe8a0b559a5b45239fb91756b451553b049
Nov 02 04:05:43.861080 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:43.860974    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"]
Nov 02 04:05:44.499276 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:44.499241    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx" event={"ID":"0f37870b-0726-49e6-92ea-a55914f57f2b","Type":"ContainerStarted","Data":"f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1"}
Nov 02 04:05:44.499468 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:44.499303    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx" event={"ID":"0f37870b-0726-49e6-92ea-a55914f57f2b","Type":"ContainerStarted","Data":"3aa7af85b841c1624cff69829e824e960c5c7751e04b61acec4f5d556f52d5af"}
Nov 02 04:05:44.499468 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:44.499364    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:44.501034 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:44.501005    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt" event={"ID":"7029ba4f-93aa-4d8c-aba4-b1fdaf275526","Type":"ContainerStarted","Data":"b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3"}
Nov 02 04:05:44.501163 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:44.501053    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt" event={"ID":"7029ba4f-93aa-4d8c-aba4-b1fdaf275526","Type":"ContainerStarted","Data":"7407ae2e26aa0232fe64b15c3930ffe8a0b559a5b45239fb91756b451553b049"}
Nov 02 04:05:44.501163 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:44.501144    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:44.616940 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:44.616882    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx" podStartSLOduration=2.616869569 podStartE2EDuration="2.616869569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:05:44.615470006 +0000 UTC m=+4688.410818572" watchObservedRunningTime="2026-11-02 04:05:44.616869569 +0000 UTC m=+4688.412218137"
Nov 02 04:05:45.679571 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:45.679544    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:47.659472 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:47.659429    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:48.110865 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:48.110833    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:49.292644 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:49.292575    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:49.461455 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:49.461423    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:49.582443 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:49.582347    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt" podStartSLOduration=6.582333638 podStartE2EDuration="6.582333638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:05:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:05:44.731535501 +0000 UTC m=+4688.526884069" watchObservedRunningTime="2026-11-02 04:05:49.582333638 +0000 UTC m=+4693.377682203"
Nov 02 04:05:49.583171 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:49.583152    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"]
Nov 02 04:05:49.583380 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:49.583358    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" containerID="cri-o://ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937" gracePeriod=30
Nov 02 04:05:49.640724 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:49.640691    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:51.942541 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:51.942509    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"]
Nov 02 04:05:51.943023 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:51.942789    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx" podUID="1f404c24-4402-4b99-8e03-a3818d6032ec" containerName="maas-api" containerID="cri-o://391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7" gracePeriod=30
Nov 02 04:05:52.078600 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.078570    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:52.125555 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.125523    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"]
Nov 02 04:05:52.130023 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.129999    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.184071 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.183917    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 04:05:52.184071 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.184038    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9z5bk\""
Nov 02 04:05:52.185471 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.185449    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"]
Nov 02 04:05:52.200970 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.200897    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 04:05:52.318573 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.318540    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-metrics-certs\") pod \"1f404c24-4402-4b99-8e03-a3818d6032ec\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") "
Nov 02 04:05:52.318776 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.318707    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f404c24-4402-4b99-8e03-a3818d6032ec-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f404c24-4402-4b99-8e03-a3818d6032ec-openshift-service-ca\") pod \"1f404c24-4402-4b99-8e03-a3818d6032ec\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") "
Nov 02 04:05:52.318823 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.318783    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f404c24-4402-4b99-8e03-a3818d6032ec-kube-api-access-xcmsw\" (UniqueName: \"kubernetes.io/projected/1f404c24-4402-4b99-8e03-a3818d6032ec-kube-api-access-xcmsw\") pod \"1f404c24-4402-4b99-8e03-a3818d6032ec\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") "
Nov 02 04:05:52.318871 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.318817    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls\") pod \"1f404c24-4402-4b99-8e03-a3818d6032ec\" (UID: \"1f404c24-4402-4b99-8e03-a3818d6032ec\") "
Nov 02 04:05:52.319098 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.319069    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.319284 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.319264    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdz8v\" (UniqueName: \"kubernetes.io/projected/b1c97893-5ded-49f6-9781-aa316c489320-kube-api-access-tdz8v\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.319361 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.319304    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-xks-ca\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.319361 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.319167    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f404c24-4402-4b99-8e03-a3818d6032ec-openshift-service-ca" pod "1f404c24-4402-4b99-8e03-a3818d6032ec" (UID: "1f404c24-4402-4b99-8e03-a3818d6032ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:05:52.319834 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.319811    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f404c24-4402-4b99-8e03-a3818d6032ec-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:52.320909 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.320885    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-metrics-certs" pod "1f404c24-4402-4b99-8e03-a3818d6032ec" (UID: "1f404c24-4402-4b99-8e03-a3818d6032ec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:52.321036 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.321021    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls" pod "1f404c24-4402-4b99-8e03-a3818d6032ec" (UID: "1f404c24-4402-4b99-8e03-a3818d6032ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:52.321093 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.321079    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f404c24-4402-4b99-8e03-a3818d6032ec-kube-api-access-xcmsw" pod "1f404c24-4402-4b99-8e03-a3818d6032ec" (UID: "1f404c24-4402-4b99-8e03-a3818d6032ec"). InnerVolumeSpecName "kube-api-access-xcmsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:52.420950 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.420911    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.421116 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.420961    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdz8v\" (UniqueName: \"kubernetes.io/projected/b1c97893-5ded-49f6-9781-aa316c489320-kube-api-access-tdz8v\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.421116 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.421078    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-xks-ca\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.421220 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.421207    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcmsw\" (UniqueName: \"kubernetes.io/projected/1f404c24-4402-4b99-8e03-a3818d6032ec-kube-api-access-xcmsw\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:52.421255 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.421224    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:52.421255 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.421234    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f404c24-4402-4b99-8e03-a3818d6032ec-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:52.421553 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.421536    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.423455 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.423438    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-xks-ca\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.479668 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.479569    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdz8v\" (UniqueName: \"kubernetes.io/projected/b1c97893-5ded-49f6-9781-aa316c489320-kube-api-access-tdz8v\") pod \"maas-api-revoke-sub-60f3081f-rpwtj\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:52.544736 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.544702    4407 generic.go:356] "Generic (PLEG): container finished" podID="1f404c24-4402-4b99-8e03-a3818d6032ec" containerID="391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7" exitCode=0
Nov 02 04:05:52.544907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.544786    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"
Nov 02 04:05:52.544907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.544798    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx" event={"ID":"1f404c24-4402-4b99-8e03-a3818d6032ec","Type":"ContainerDied","Data":"391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7"}
Nov 02 04:05:52.544907 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.544890    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx" event={"ID":"1f404c24-4402-4b99-8e03-a3818d6032ec","Type":"ContainerDied","Data":"d7d03c37809055c239bda1dadadfc4535514406e0d9d04dbaded4d1785a2ae64"}
Nov 02 04:05:52.545020 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.544918    4407 scope.go:122] "RemoveContainer" containerID="391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7"
Nov 02 04:05:52.554487 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.554462    4407 scope.go:122] "RemoveContainer" containerID="391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7"
Nov 02 04:05:52.554846 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:52.554815    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7\": container with ID starting with 391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7 not found: ID does not exist" containerID="391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7"
Nov 02 04:05:52.554947 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.554855    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7"} err="failed to get container status \"391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7\": rpc error: code = NotFound desc = could not find container \"391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7\": container with ID starting with 391727d8337acd6a6739d0fc4a1c5fb1fde58498de6cb047ec6b53650f6eabe7 not found: ID does not exist"
Nov 02 04:05:52.712318 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.712283    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"]
Nov 02 04:05:52.716264 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.716239    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-398828-58f7c6fd84-m6pcx"]
Nov 02 04:05:52.743609 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:52.743587    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:53.006938 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.006860    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:53.118026 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.117997    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"]
Nov 02 04:05:53.119779 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:05:53.119754    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c97893_5ded_49f6_9781_aa316c489320.slice/crio-e8526376ae3a8cbcf6356da1df89e75e3c7926f4722933822f2400d1c2fd1851 WatchSource:0}: Error finding container e8526376ae3a8cbcf6356da1df89e75e3c7926f4722933822f2400d1c2fd1851: Status 404 returned error can't find the container with id e8526376ae3a8cbcf6356da1df89e75e3c7926f4722933822f2400d1c2fd1851
Nov 02 04:05:53.206847 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.206821    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:53.479214 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.479176    4407 patch_prober.go:29] interesting pod/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 04:05:53.479378 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.479229    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 04:05:53.552063 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.552029    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj" event={"ID":"b1c97893-5ded-49f6-9781-aa316c489320","Type":"ContainerStarted","Data":"b63439f160a2004adce17afbfe84dd052ca4dbc01c2d366ec90a49a92d80ffbd"}
Nov 02 04:05:53.552238 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.552083    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj" event={"ID":"b1c97893-5ded-49f6-9781-aa316c489320","Type":"ContainerStarted","Data":"e8526376ae3a8cbcf6356da1df89e75e3c7926f4722933822f2400d1c2fd1851"}
Nov 02 04:05:53.642665 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.642544    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:53.666648 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.666572    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj" podStartSLOduration=1.66655815 podStartE2EDuration="1.66655815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:05:53.666329499 +0000 UTC m=+4697.461678065" watchObservedRunningTime="2026-11-02 04:05:53.66655815 +0000 UTC m=+4697.461906746"
Nov 02 04:05:53.872815 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:53.872783    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:54.046127 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.046095    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:54.351210 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.351176    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f404c24-4402-4b99-8e03-a3818d6032ec" path="/var/lib/kubelet/pods/1f404c24-4402-4b99-8e03-a3818d6032ec/volumes"
Nov 02 04:05:54.541374 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.541339    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:54.559761 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.559729    4407 generic.go:356] "Generic (PLEG): container finished" podID="b1c97893-5ded-49f6-9781-aa316c489320" containerID="b63439f160a2004adce17afbfe84dd052ca4dbc01c2d366ec90a49a92d80ffbd" exitCode=0
Nov 02 04:05:54.559924 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.559814    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj" event={"ID":"b1c97893-5ded-49f6-9781-aa316c489320","Type":"ContainerDied","Data":"b63439f160a2004adce17afbfe84dd052ca4dbc01c2d366ec90a49a92d80ffbd"}
Nov 02 04:05:54.845170 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.845145    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 04:05:54.848232 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848214    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0aeab29-37eb-4500-bbb6-79543a341348-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aeab29-37eb-4500-bbb6-79543a341348-istio-podinfo\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848321 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848250    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-credential-socket\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848321 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848292    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-certs\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848321 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848312    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0aeab29-37eb-4500-bbb6-79543a341348-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aeab29-37eb-4500-bbb6-79543a341348-istiod-ca-cert\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848461 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848340    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-socket\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848461 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848370    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-istio-token\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-istio-token\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848461 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848409    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-data\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848461 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848437    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-kube-api-access-ssk9l\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-kube-api-access-ssk9l\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848597 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848513    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-envoy\") pod \"f0aeab29-37eb-4500-bbb6-79543a341348\" (UID: \"f0aeab29-37eb-4500-bbb6-79543a341348\") "
Nov 02 04:05:54.848688 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848639    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-certs" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:54.848901 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848882    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:54.849040 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848799    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-data" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:54.849040 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.848868    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-socket" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:54.849040 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.849003    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-credential-socket" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:54.849040 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.849013    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0aeab29-37eb-4500-bbb6-79543a341348-istiod-ca-cert" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:05:54.850643 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.850604    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0aeab29-37eb-4500-bbb6-79543a341348-istio-podinfo" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:05:54.850744 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.850707    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-istio-token" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:54.851071 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.851039    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-kube-api-access-ssk9l" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "kube-api-access-ssk9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:54.851071 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.851057    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-envoy" pod "f0aeab29-37eb-4500-bbb6-79543a341348" (UID: "f0aeab29-37eb-4500-bbb6-79543a341348"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:05:54.867031 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.867004    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:54.949435 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.949405    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:54.949435 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.949434    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:54.949661 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.949447    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssk9l\" (UniqueName: \"kubernetes.io/projected/f0aeab29-37eb-4500-bbb6-79543a341348-kube-api-access-ssk9l\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:54.949661 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.949461    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:54.949661 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.949473    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aeab29-37eb-4500-bbb6-79543a341348-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:54.949661 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.949494    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:54.949661 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.949505    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aeab29-37eb-4500-bbb6-79543a341348-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:54.949661 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:54.949518    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aeab29-37eb-4500-bbb6-79543a341348-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:55.509972 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.509945    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:05:55.533502 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.510009    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:05:55.566460 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.566425    4407 generic.go:356] "Generic (PLEG): container finished" podID="f0aeab29-37eb-4500-bbb6-79543a341348" containerID="ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937" exitCode=0
Nov 02 04:05:55.566654 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.566477    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"
Nov 02 04:05:55.566654 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.566516    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" event={"ID":"f0aeab29-37eb-4500-bbb6-79543a341348","Type":"ContainerDied","Data":"ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937"}
Nov 02 04:05:55.566654 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.566575    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q" event={"ID":"f0aeab29-37eb-4500-bbb6-79543a341348","Type":"ContainerDied","Data":"382b6890e8b46e9ac162d493577d63d12b63a66986c1547b8caf0e0eeff2f754"}
Nov 02 04:05:55.566654 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.566605    4407 scope.go:122] "RemoveContainer" containerID="ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937"
Nov 02 04:05:55.576870 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.576851    4407 scope.go:122] "RemoveContainer" containerID="ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937"
Nov 02 04:05:55.577162 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:05:55.577141    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937\": container with ID starting with ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937 not found: ID does not exist" containerID="ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937"
Nov 02 04:05:55.577220 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.577170    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937"} err="failed to get container status \"ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937\": rpc error: code = NotFound desc = could not find container \"ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937\": container with ID starting with ac83a5a154f9f82c73b2bd54df09efeebce52943fb3fb1f714217522ee4af937 not found: ID does not exist"
Nov 02 04:05:55.711824 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.711802    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:55.757411 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.757383    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-xks-ca\") pod \"b1c97893-5ded-49f6-9781-aa316c489320\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") "
Nov 02 04:05:55.757617 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.757443    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-openshift-ca\") pod \"b1c97893-5ded-49f6-9781-aa316c489320\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") "
Nov 02 04:05:55.757617 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.757503    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1c97893-5ded-49f6-9781-aa316c489320-kube-api-access-tdz8v\" (UniqueName: \"kubernetes.io/projected/b1c97893-5ded-49f6-9781-aa316c489320-kube-api-access-tdz8v\") pod \"b1c97893-5ded-49f6-9781-aa316c489320\" (UID: \"b1c97893-5ded-49f6-9781-aa316c489320\") "
Nov 02 04:05:55.758240 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.758195    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-openshift-ca" pod "b1c97893-5ded-49f6-9781-aa316c489320" (UID: "b1c97893-5ded-49f6-9781-aa316c489320"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:05:55.760113 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.760056    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1c97893-5ded-49f6-9781-aa316c489320-kube-api-access-tdz8v" pod "b1c97893-5ded-49f6-9781-aa316c489320" (UID: "b1c97893-5ded-49f6-9781-aa316c489320"). InnerVolumeSpecName "kube-api-access-tdz8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:05:55.760245 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.760218    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-xks-ca" pod "b1c97893-5ded-49f6-9781-aa316c489320" (UID: "b1c97893-5ded-49f6-9781-aa316c489320"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:05:55.857380 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.857345    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"]
Nov 02 04:05:55.858524 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.858497    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-xks-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:55.858524 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.858522    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1c97893-5ded-49f6-9781-aa316c489320-maas-api-openshift-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:55.858905 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.858534    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdz8v\" (UniqueName: \"kubernetes.io/projected/b1c97893-5ded-49f6-9781-aa316c489320-kube-api-access-tdz8v\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:05:55.908015 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:55.907985    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ca6546-openshift-default-994bcc7f5-xpw6q"]
Nov 02 04:05:56.350289 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:56.350248    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" path="/var/lib/kubelet/pods/f0aeab29-37eb-4500-bbb6-79543a341348/volumes"
Nov 02 04:05:56.572323 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:56.572296    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"
Nov 02 04:05:56.572780 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:56.572335    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj" event={"ID":"b1c97893-5ded-49f6-9781-aa316c489320","Type":"ContainerDied","Data":"e8526376ae3a8cbcf6356da1df89e75e3c7926f4722933822f2400d1c2fd1851"}
Nov 02 04:05:56.572780 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:56.572419    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8526376ae3a8cbcf6356da1df89e75e3c7926f4722933822f2400d1c2fd1851"
Nov 02 04:05:58.641972 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:58.641942    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:05:58.982233 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:05:58.982203    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:00.787794 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:00.787759    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:00.980219 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:00.980185    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:01.080381 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:01.080298    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"]
Nov 02 04:06:01.080586 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:01.080561    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy" containerID="cri-o://5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d" gracePeriod=30
Nov 02 04:06:01.152006 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:01.151945    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:03.784528 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:03.784499    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:04.675519 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:04.675486    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:05.190814 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:06:05.190780    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:06:06.338345 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.338319    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 04:06:06.361523 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361444    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-certs\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.361678 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361524    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-credential-socket\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.361678 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361554    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-kube-api-access-t4rkz\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-kube-api-access-t4rkz\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.361678 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361587    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-socket\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.361678 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361615    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-envoy\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.361903 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361714    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-podinfo\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.361903 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361726    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-certs" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:06:06.361903 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361811    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-credential-socket" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:06:06.361903 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361838    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-token\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-token\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.361903 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361862    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7086420f-22a5-4dff-b91c-fa81c4d4e279-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7086420f-22a5-4dff-b91c-fa81c4d4e279-istiod-ca-cert\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.361903 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.361894    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-data\") pod \"7086420f-22a5-4dff-b91c-fa81c4d4e279\" (UID: \"7086420f-22a5-4dff-b91c-fa81c4d4e279\") "
Nov 02 04:06:06.362219 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.362002    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-socket" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:06:06.362219 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.362191    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-data" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:06:06.362311 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.362221    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7086420f-22a5-4dff-b91c-fa81c4d4e279-istiod-ca-cert" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:06:06.362369 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.362351    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7086420f-22a5-4dff-b91c-fa81c4d4e279-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.362422 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.362375    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.362422 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.362391    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.362511 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.362430    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.362593 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.362574    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.364009 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.363986    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-podinfo" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:06:06.364120 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.364023    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-token" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:06:06.364364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.364348    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-envoy" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:06:06.364364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.364354    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-kube-api-access-t4rkz" pod "7086420f-22a5-4dff-b91c-fa81c4d4e279" (UID: "7086420f-22a5-4dff-b91c-fa81c4d4e279"). InnerVolumeSpecName "kube-api-access-t4rkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:06:06.463597 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.463561    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.463597 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.463588    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.463597 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.463597    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4rkz\" (UniqueName: \"kubernetes.io/projected/7086420f-22a5-4dff-b91c-fa81c4d4e279-kube-api-access-t4rkz\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.463597 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.463607    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7086420f-22a5-4dff-b91c-fa81c4d4e279-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:06.625636 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.625542    4407 generic.go:356] "Generic (PLEG): container finished" podID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerID="5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d" exitCode=0
Nov 02 04:06:06.625636 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.625590    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"
Nov 02 04:06:06.625859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.625643    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" event={"ID":"7086420f-22a5-4dff-b91c-fa81c4d4e279","Type":"ContainerDied","Data":"5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d"}
Nov 02 04:06:06.625859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.625695    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f" event={"ID":"7086420f-22a5-4dff-b91c-fa81c4d4e279","Type":"ContainerDied","Data":"7a0415672f7b5051de9ddee89abe76f3bf97da74909d559df2d49b82796d49c7"}
Nov 02 04:06:06.625859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.625723    4407 scope.go:122] "RemoveContainer" containerID="5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d"
Nov 02 04:06:06.635173 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.635159    4407 scope.go:122] "RemoveContainer" containerID="17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3"
Nov 02 04:06:06.644158 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.644140    4407 scope.go:122] "RemoveContainer" containerID="5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d"
Nov 02 04:06:06.644387 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:06:06.644370    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d\": container with ID starting with 5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d not found: ID does not exist" containerID="5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d"
Nov 02 04:06:06.644445 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.644396    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d"} err="failed to get container status \"5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d\": rpc error: code = NotFound desc = could not find container \"5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d\": container with ID starting with 5dcf41e990a8ab8e90f7bbe0313e001328f314ceffffdeecf568fb629e25336d not found: ID does not exist"
Nov 02 04:06:06.644445 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.644420    4407 scope.go:122] "RemoveContainer" containerID="17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3"
Nov 02 04:06:06.644719 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:06:06.644701    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3\": container with ID starting with 17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3 not found: ID does not exist" containerID="17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3"
Nov 02 04:06:06.644787 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.644726    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3"} err="failed to get container status \"17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3\": rpc error: code = NotFound desc = could not find container \"17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3\": container with ID starting with 17b1e7132bf804c12dfa5ce51914e5eaeb2bd15520f54c7676b02018b53007d3 not found: ID does not exist"
Nov 02 04:06:06.793489 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.793449    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"]
Nov 02 04:06:06.797618 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:06.797597    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-398828-openshift-default-5647988965-ph22f"]
Nov 02 04:06:07.920457 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:07.920424    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:06:08.351309 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:08.351273    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" path="/var/lib/kubelet/pods/7086420f-22a5-4dff-b91c-fa81c4d4e279/volumes"
Nov 02 04:06:08.597893 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:08.597860    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:08.758371 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:08.758346    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:08.978885 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:08.978845    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:10.347744 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:10.347705    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:10.515941 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:10.515910    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:12.561592 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:12.561567    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:12.725725 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:12.725690    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:13.674758 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:13.674725    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:18.470193 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:18.470106    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:22.496894 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.496861    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"]
Nov 02 04:06:22.497350 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.497064    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5" podUID="14dba853-b2ce-4244-aa40-e1cea64d5af0" containerName="maas-api" containerID="cri-o://8e94b85297acf5187caa2691c2e2d37e53a2a83c39449e6e0ece7662b3ebaf96" gracePeriod=30
Nov 02 04:06:22.620835 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.620805    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:22.712212 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.711920    4407 generic.go:356] "Generic (PLEG): container finished" podID="14dba853-b2ce-4244-aa40-e1cea64d5af0" containerID="8e94b85297acf5187caa2691c2e2d37e53a2a83c39449e6e0ece7662b3ebaf96" exitCode=0
Nov 02 04:06:22.712212 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.712049    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5" event={"ID":"14dba853-b2ce-4244-aa40-e1cea64d5af0","Type":"ContainerDied","Data":"8e94b85297acf5187caa2691c2e2d37e53a2a83c39449e6e0ece7662b3ebaf96"}
Nov 02 04:06:22.752962 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.752907    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 04:06:22.824674 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.824644    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14dba853-b2ce-4244-aa40-e1cea64d5af0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dba853-b2ce-4244-aa40-e1cea64d5af0-openshift-service-ca\") pod \"14dba853-b2ce-4244-aa40-e1cea64d5af0\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") "
Nov 02 04:06:22.824674 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.824676    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-metrics-certs\") pod \"14dba853-b2ce-4244-aa40-e1cea64d5af0\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") "
Nov 02 04:06:22.824940 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.824729    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-maas-api-tls\") pod \"14dba853-b2ce-4244-aa40-e1cea64d5af0\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") "
Nov 02 04:06:22.824940 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.824898    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14dba853-b2ce-4244-aa40-e1cea64d5af0-kube-api-access-x27ns\" (UniqueName: \"kubernetes.io/projected/14dba853-b2ce-4244-aa40-e1cea64d5af0-kube-api-access-x27ns\") pod \"14dba853-b2ce-4244-aa40-e1cea64d5af0\" (UID: \"14dba853-b2ce-4244-aa40-e1cea64d5af0\") "
Nov 02 04:06:22.825083 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.825055    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14dba853-b2ce-4244-aa40-e1cea64d5af0-openshift-service-ca" pod "14dba853-b2ce-4244-aa40-e1cea64d5af0" (UID: "14dba853-b2ce-4244-aa40-e1cea64d5af0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:06:22.825290 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.825273    4407 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dba853-b2ce-4244-aa40-e1cea64d5af0-openshift-service-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:22.827017 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.826987    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-maas-api-tls" pod "14dba853-b2ce-4244-aa40-e1cea64d5af0" (UID: "14dba853-b2ce-4244-aa40-e1cea64d5af0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:06:22.827132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.827054    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-metrics-certs" pod "14dba853-b2ce-4244-aa40-e1cea64d5af0" (UID: "14dba853-b2ce-4244-aa40-e1cea64d5af0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:06:22.827132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.827079    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14dba853-b2ce-4244-aa40-e1cea64d5af0-kube-api-access-x27ns" pod "14dba853-b2ce-4244-aa40-e1cea64d5af0" (UID: "14dba853-b2ce-4244-aa40-e1cea64d5af0"). InnerVolumeSpecName "kube-api-access-x27ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:06:22.926664 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.926610    4407 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-metrics-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:22.926664 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.926660    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14dba853-b2ce-4244-aa40-e1cea64d5af0-maas-api-tls\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:22.926873 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:22.926670    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x27ns\" (UniqueName: \"kubernetes.io/projected/14dba853-b2ce-4244-aa40-e1cea64d5af0-kube-api-access-x27ns\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:06:23.717829 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:23.717783    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5" event={"ID":"14dba853-b2ce-4244-aa40-e1cea64d5af0","Type":"ContainerDied","Data":"1df94052c0dad023807000cd4e27f49fcf432749196fdac66792778d4c59e152"}
Nov 02 04:06:23.717829 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:23.717811    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"
Nov 02 04:06:23.718525 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:23.717862    4407 scope.go:122] "RemoveContainer" containerID="8e94b85297acf5187caa2691c2e2d37e53a2a83c39449e6e0ece7662b3ebaf96"
Nov 02 04:06:23.884806 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:23.884770    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"]
Nov 02 04:06:23.938109 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:23.938072    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5ac93f-7d9ddb766d-4zls5"]
Nov 02 04:06:24.148608 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:24.148555    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:24.350225 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:24.350194    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14dba853-b2ce-4244-aa40-e1cea64d5af0" path="/var/lib/kubelet/pods/14dba853-b2ce-4244-aa40-e1cea64d5af0/volumes"
Nov 02 04:06:24.369432 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:24.369407    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:24.526179 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:24.526139    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:25.024533 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:25.024506    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:29.405075 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:29.405043    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:30.955714 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:30.955685    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:31.123330 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:31.123292    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:31.306711 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:31.306669    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:34.200652 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:34.196453    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:35.210890 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:06:35.210848    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:06:38.496780 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:38.496726    4407 patch_prober.go:29] interesting pod/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 04:06:38.497319 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:38.496796    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 04:06:38.935814 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:38.935783    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:06:39.066125 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:39.066082    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:39.801202 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:39.801172    4407 generic.go:356] "Generic (PLEG): container finished" podID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerID="4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c" exitCode=0
Nov 02 04:06:39.801601 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:39.801240    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" event={"ID":"bdf2cf02-113e-4472-a59d-30abda30c9d3","Type":"ContainerDied","Data":"4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c"}
Nov 02 04:06:39.801601 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:39.801579    4407 scope.go:122] "RemoveContainer" containerID="4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c"
Nov 02 04:06:39.802707 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:39.802678    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:06:39.802773 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:39.802747    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:06:39.802809 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:39.802782    4407 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 02 04:06:40.812370 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:40.812332    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" event={"ID":"bdf2cf02-113e-4472-a59d-30abda30c9d3","Type":"ContainerStarted","Data":"ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f"}
Nov 02 04:06:41.100706 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:41.099071    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:06:41.100706 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:41.099208    4407 patch_prober.go:29] interesting pod/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Nov 02 04:06:41.100706 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:41.099267    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Nov 02 04:06:42.099652 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:42.091039    4407 patch_prober.go:29] interesting pod/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Nov 02 04:06:42.099652 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:42.091104    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Nov 02 04:06:43.098095 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:43.098057    4407 patch_prober.go:29] interesting pod/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Nov 02 04:06:43.098413 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:43.098383    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Nov 02 04:06:43.906581 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:43.906542    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:44.099652 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:44.099083    4407 patch_prober.go:29] interesting pod/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Nov 02 04:06:44.099652 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:44.099148    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Nov 02 04:06:45.099117 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:45.099072    4407 patch_prober.go:29] interesting pod/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Nov 02 04:06:45.099772 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:45.099738    4407 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Nov 02 04:06:45.117930 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:45.117893    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:46.095092 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:46.095060    4407 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:06:46.095472 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:46.095458    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:06:46.096344 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:46.096325    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:06:46.196137 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:46.196107    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:46.350956 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:46.350871    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:46.520891 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:46.520844    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:46.731549 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:46.731468    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:46.914035 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:46.914006    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:47.678841 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:47.678811    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:47.839100 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:47.839065    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:48.975889 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:48.975839    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:52.584750 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:52.584718    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:06:53.785790 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:53.785758    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:55.185279 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:55.185249    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:56.252822 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:56.252773    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:56.407009 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:56.406973    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:58.571946 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:58.571885    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:59.442331 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:59.442296    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:06:59.599017 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:06:59.598985    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:00.160903 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:00.160872    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:03.366328 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:03.366288    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:05.187447 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:07:05.187407    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:07:06.143947 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:06.143917    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:07:06.312765 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:06.312734    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:06.479418 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:06.479325    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:08.189869 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:08.189824    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:12.278734 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.278703    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:12.448614 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.448583    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:12.795770 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.795736    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"]
Nov 02 04:07:12.796158 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796145    4407 cpu_manager.go:414] "containerMap: removing container" podUID="1f404c24-4402-4b99-8e03-a3818d6032ec" containerName="maas-api"
Nov 02 04:07:12.796210 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796159    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f404c24-4402-4b99-8e03-a3818d6032ec" containerName="maas-api"
Nov 02 04:07:12.796210 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796170    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy"
Nov 02 04:07:12.796210 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796175    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy"
Nov 02 04:07:12.796210 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796189    4407 cpu_manager.go:414] "containerMap: removing container" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy"
Nov 02 04:07:12.796210 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796194    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy"
Nov 02 04:07:12.796210 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796204    4407 cpu_manager.go:414] "containerMap: removing container" podUID="14dba853-b2ce-4244-aa40-e1cea64d5af0" containerName="maas-api"
Nov 02 04:07:12.796210 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796209    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14dba853-b2ce-4244-aa40-e1cea64d5af0" containerName="maas-api"
Nov 02 04:07:12.796436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796230    4407 cpu_manager.go:414] "containerMap: removing container" podUID="b1c97893-5ded-49f6-9781-aa316c489320" containerName="revoke-keys"
Nov 02 04:07:12.796436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796235    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1c97893-5ded-49f6-9781-aa316c489320" containerName="revoke-keys"
Nov 02 04:07:12.796436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796300    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy"
Nov 02 04:07:12.796436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796306    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1c97893-5ded-49f6-9781-aa316c489320" containerName="revoke-keys"
Nov 02 04:07:12.796436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796315    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f404c24-4402-4b99-8e03-a3818d6032ec" containerName="maas-api"
Nov 02 04:07:12.796436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796323    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0aeab29-37eb-4500-bbb6-79543a341348" containerName="istio-proxy"
Nov 02 04:07:12.796436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.796329    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="14dba853-b2ce-4244-aa40-e1cea64d5af0" containerName="maas-api"
Nov 02 04:07:12.799456 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.799441    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:12.855541 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.855508    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"]
Nov 02 04:07:12.907853 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.907824    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 04:07:12.908113 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.908094    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9z5bk\""
Nov 02 04:07:12.933512 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.933479    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:12.933679 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.933537    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:12.933679 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:12.933616    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qltbd\" (UniqueName: \"kubernetes.io/projected/b7797f59-b12f-43a0-949d-7bad545ec127-kube-api-access-qltbd\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:13.010933 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.010901    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:13.034695 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.034667    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:13.034868 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.034710    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:13.034868 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.034736    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qltbd\" (UniqueName: \"kubernetes.io/projected/b7797f59-b12f-43a0-949d-7bad545ec127-kube-api-access-qltbd\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:13.035368 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.035346    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:13.037216 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.037195    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:13.093177 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.093087    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qltbd\" (UniqueName: \"kubernetes.io/projected/b7797f59-b12f-43a0-949d-7bad545ec127-kube-api-access-qltbd\") pod \"maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:13.112047 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.112017    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:13.484316 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.484287    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"]
Nov 02 04:07:13.485045 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:07:13.485011    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7797f59_b12f_43a0_949d_7bad545ec127.slice/crio-dc0a23a4bd505a1ab6c38ea551a4a738fc8606e25c289c1792f0b73f439735c1 WatchSource:0}: Error finding container dc0a23a4bd505a1ab6c38ea551a4a738fc8606e25c289c1792f0b73f439735c1: Status 404 returned error can't find the container with id dc0a23a4bd505a1ab6c38ea551a4a738fc8606e25c289c1792f0b73f439735c1
Nov 02 04:07:13.978183 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.978148    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n" event={"ID":"b7797f59-b12f-43a0-949d-7bad545ec127","Type":"ContainerStarted","Data":"13ee9e8ab93d9f41a7098e9ca55f0ff23c11d5d3c317f83cd0b759f5a712e934"}
Nov 02 04:07:13.978380 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:13.978236    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n" event={"ID":"b7797f59-b12f-43a0-949d-7bad545ec127","Type":"ContainerStarted","Data":"dc0a23a4bd505a1ab6c38ea551a4a738fc8606e25c289c1792f0b73f439735c1"}
Nov 02 04:07:14.092561 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:14.092465    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n" podStartSLOduration=2.092450885 podStartE2EDuration="2.092450885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:07:14.092203493 +0000 UTC m=+4777.887552059" watchObservedRunningTime="2026-11-02 04:07:14.092450885 +0000 UTC m=+4777.887799450"
Nov 02 04:07:14.984274 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:14.984243    4407 generic.go:356] "Generic (PLEG): container finished" podID="b7797f59-b12f-43a0-949d-7bad545ec127" containerID="13ee9e8ab93d9f41a7098e9ca55f0ff23c11d5d3c317f83cd0b759f5a712e934" exitCode=0
Nov 02 04:07:14.984669 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:14.984295    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n" event={"ID":"b7797f59-b12f-43a0-949d-7bad545ec127","Type":"ContainerDied","Data":"13ee9e8ab93d9f41a7098e9ca55f0ff23c11d5d3c317f83cd0b759f5a712e934"}
Nov 02 04:07:16.126086 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.126060    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:16.163257 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.163230    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-xks-ca\") pod \"b7797f59-b12f-43a0-949d-7bad545ec127\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") "
Nov 02 04:07:16.163396 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.163278    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-openshift-ca\") pod \"b7797f59-b12f-43a0-949d-7bad545ec127\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") "
Nov 02 04:07:16.163396 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.163372    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7797f59-b12f-43a0-949d-7bad545ec127-kube-api-access-qltbd\" (UniqueName: \"kubernetes.io/projected/b7797f59-b12f-43a0-949d-7bad545ec127-kube-api-access-qltbd\") pod \"b7797f59-b12f-43a0-949d-7bad545ec127\" (UID: \"b7797f59-b12f-43a0-949d-7bad545ec127\") "
Nov 02 04:07:16.163681 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.163662    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-openshift-ca" pod "b7797f59-b12f-43a0-949d-7bad545ec127" (UID: "b7797f59-b12f-43a0-949d-7bad545ec127"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:07:16.165549 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.165521    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-xks-ca" pod "b7797f59-b12f-43a0-949d-7bad545ec127" (UID: "b7797f59-b12f-43a0-949d-7bad545ec127"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:07:16.165667 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.165549    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7797f59-b12f-43a0-949d-7bad545ec127-kube-api-access-qltbd" pod "b7797f59-b12f-43a0-949d-7bad545ec127" (UID: "b7797f59-b12f-43a0-949d-7bad545ec127"). InnerVolumeSpecName "kube-api-access-qltbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:07:16.264309 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.264276    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qltbd\" (UniqueName: \"kubernetes.io/projected/b7797f59-b12f-43a0-949d-7bad545ec127-kube-api-access-qltbd\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:07:16.264309 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.264309    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-xks-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:07:16.264531 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.264325    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7797f59-b12f-43a0-949d-7bad545ec127-maas-api-openshift-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:07:16.995234 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.995196    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"
Nov 02 04:07:16.995234 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.995211    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n" event={"ID":"b7797f59-b12f-43a0-949d-7bad545ec127","Type":"ContainerDied","Data":"dc0a23a4bd505a1ab6c38ea551a4a738fc8606e25c289c1792f0b73f439735c1"}
Nov 02 04:07:16.995468 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:16.995267    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc0a23a4bd505a1ab6c38ea551a4a738fc8606e25c289c1792f0b73f439735c1"
Nov 02 04:07:18.039364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:18.039335    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:22.776498 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:22.776466    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:24.364367 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:24.364335    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"]
Nov 02 04:07:24.641366 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:24.641281    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:24.789614 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:24.789579    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:27.474382 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:27.474343    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:29.365000 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:29.364956    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9" podUID="b005c7c1-638c-4ed0-bfa3-21148da14b61" containerName="payload-pre-processing" containerID="cri-o://4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d" gracePeriod=25
Nov 02 04:07:29.616098 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:29.616042    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:07:29.699411 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:29.699384    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b005c7c1-638c-4ed0-bfa3-21148da14b61-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b005c7c1-638c-4ed0-bfa3-21148da14b61-plugins-config-volume\") pod \"b005c7c1-638c-4ed0-bfa3-21148da14b61\" (UID: \"b005c7c1-638c-4ed0-bfa3-21148da14b61\") "
Nov 02 04:07:29.699605 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:29.699419    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b005c7c1-638c-4ed0-bfa3-21148da14b61-kube-api-access-lfbck\" (UniqueName: \"kubernetes.io/projected/b005c7c1-638c-4ed0-bfa3-21148da14b61-kube-api-access-lfbck\") pod \"b005c7c1-638c-4ed0-bfa3-21148da14b61\" (UID: \"b005c7c1-638c-4ed0-bfa3-21148da14b61\") "
Nov 02 04:07:29.699835 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:29.699812    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b005c7c1-638c-4ed0-bfa3-21148da14b61-plugins-config-volume" pod "b005c7c1-638c-4ed0-bfa3-21148da14b61" (UID: "b005c7c1-638c-4ed0-bfa3-21148da14b61"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:07:29.701652 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:29.701618    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b005c7c1-638c-4ed0-bfa3-21148da14b61-kube-api-access-lfbck" pod "b005c7c1-638c-4ed0-bfa3-21148da14b61" (UID: "b005c7c1-638c-4ed0-bfa3-21148da14b61"). InnerVolumeSpecName "kube-api-access-lfbck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:07:29.801150 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:29.801118    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b005c7c1-638c-4ed0-bfa3-21148da14b61-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:07:29.801150 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:29.801146    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfbck\" (UniqueName: \"kubernetes.io/projected/b005c7c1-638c-4ed0-bfa3-21148da14b61-kube-api-access-lfbck\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:07:30.058589 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.058555    4407 generic.go:356] "Generic (PLEG): container finished" podID="b005c7c1-638c-4ed0-bfa3-21148da14b61" containerID="4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d" exitCode=0
Nov 02 04:07:30.058828 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.058664    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"
Nov 02 04:07:30.058828 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.058655    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9" event={"ID":"b005c7c1-638c-4ed0-bfa3-21148da14b61","Type":"ContainerDied","Data":"4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d"}
Nov 02 04:07:30.058828 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.058803    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9" event={"ID":"b005c7c1-638c-4ed0-bfa3-21148da14b61","Type":"ContainerDied","Data":"42b27c580733793e20e65f6b537a40b4dac099c82b471b7fb303083ef7d0cfab"}
Nov 02 04:07:30.058828 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.058829    4407 scope.go:122] "RemoveContainer" containerID="4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d"
Nov 02 04:07:30.068345 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.068328    4407 scope.go:122] "RemoveContainer" containerID="4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d"
Nov 02 04:07:30.068660 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:07:30.068638    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d\": container with ID starting with 4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d not found: ID does not exist" containerID="4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d"
Nov 02 04:07:30.068729 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.068672    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d"} err="failed to get container status \"4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d\": rpc error: code = NotFound desc = could not find container \"4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d\": container with ID starting with 4763a2cbfd08de8c35596eba985a578f80ffd2d271a9169c8cf500d909e2292d not found: ID does not exist"
Nov 02 04:07:30.225311 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.225278    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"]
Nov 02 04:07:30.275930 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.275897    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3f24d0-5cb96cdc8c-6p6b9"]
Nov 02 04:07:30.351323 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:30.351246    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b005c7c1-638c-4ed0-bfa3-21148da14b61" path="/var/lib/kubelet/pods/b005c7c1-638c-4ed0-bfa3-21148da14b61/volumes"
Nov 02 04:07:32.274778 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:32.274745    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:35.188881 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:07:35.188841    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:07:37.076145 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:37.076110    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:41.874264 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:41.874232    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:46.666426 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:46.666396    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:50.831386 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:50.831311    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:52.358768 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:52.358736    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:52.571663 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:52.571606    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:52.734288 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:52.734200    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:52.838478 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:52.838447    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"]
Nov 02 04:07:52.859816 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:52.859777    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-a-3f24d0-zvf9n"]
Nov 02 04:07:53.271965 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:53.271926    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:54.350176 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:54.350143    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7797f59-b12f-43a0-949d-7bad545ec127" path="/var/lib/kubelet/pods/b7797f59-b12f-43a0-949d-7bad545ec127/volumes"
Nov 02 04:07:55.611980 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:55.611946    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:56.569800 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:56.569762    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:07:57.703819 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:57.703782    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:58.988497 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:58.988462    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:59.150017 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:59.149984    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:07:59.320655 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:07:59.320586    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:00.074152 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074119    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"]
Nov 02 04:08:00.074556 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074521    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy"
Nov 02 04:08:00.074556 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074532    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy"
Nov 02 04:08:00.074556 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074554    4407 cpu_manager.go:414] "containerMap: removing container" podUID="b7797f59-b12f-43a0-949d-7bad545ec127" containerName="revoke-keys"
Nov 02 04:08:00.074777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074559    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7797f59-b12f-43a0-949d-7bad545ec127" containerName="revoke-keys"
Nov 02 04:08:00.074777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074574    4407 cpu_manager.go:414] "containerMap: removing container" podUID="b005c7c1-638c-4ed0-bfa3-21148da14b61" containerName="payload-pre-processing"
Nov 02 04:08:00.074777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074579    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b005c7c1-638c-4ed0-bfa3-21148da14b61" containerName="payload-pre-processing"
Nov 02 04:08:00.074777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074663    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7797f59-b12f-43a0-949d-7bad545ec127" containerName="revoke-keys"
Nov 02 04:08:00.074777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074674    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7086420f-22a5-4dff-b91c-fa81c4d4e279" containerName="istio-proxy"
Nov 02 04:08:00.074777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.074680    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="b005c7c1-638c-4ed0-bfa3-21148da14b61" containerName="payload-pre-processing"
Nov 02 04:08:00.077996 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.077979    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.135741 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.135712    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"]
Nov 02 04:08:00.183499 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.183466    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 04:08:00.183698 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.183509    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9z5bk\""
Nov 02 04:08:00.193656 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.193608    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.193768 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.193755    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.193836 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.193821    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25778\" (UniqueName: \"kubernetes.io/projected/389d6a5d-c283-47de-8286-a82d32bab4d6-kube-api-access-25778\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.295342 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.295305    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.295560 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.295369    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25778\" (UniqueName: \"kubernetes.io/projected/389d6a5d-c283-47de-8286-a82d32bab4d6-kube-api-access-25778\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.295560 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.295432    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.296005 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.295983    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.297847 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.297824    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.353307 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.353244    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25778\" (UniqueName: \"kubernetes.io/projected/389d6a5d-c283-47de-8286-a82d32bab4d6-kube-api-access-25778\") pod \"maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.390178 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.390149    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:00.760161 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:00.760128    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"]
Nov 02 04:08:00.761543 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:08:00.761508    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389d6a5d_c283_47de_8286_a82d32bab4d6.slice/crio-ae3b3e44849740ed49c84741b2124bcd95b6e47dd2a19bca35a10fb620b26b3f WatchSource:0}: Error finding container ae3b3e44849740ed49c84741b2124bcd95b6e47dd2a19bca35a10fb620b26b3f: Status 404 returned error can't find the container with id ae3b3e44849740ed49c84741b2124bcd95b6e47dd2a19bca35a10fb620b26b3f
Nov 02 04:08:01.207694 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:01.207662    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw" event={"ID":"389d6a5d-c283-47de-8286-a82d32bab4d6","Type":"ContainerStarted","Data":"c8420820f044f572ece10074d76e59060ed1549c930fdb967aaa7acc2b110bb7"}
Nov 02 04:08:01.208096 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:01.207714    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw" event={"ID":"389d6a5d-c283-47de-8286-a82d32bab4d6","Type":"ContainerStarted","Data":"ae3b3e44849740ed49c84741b2124bcd95b6e47dd2a19bca35a10fb620b26b3f"}
Nov 02 04:08:01.323410 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:01.323357    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw" podStartSLOduration=1.323342921 podStartE2EDuration="1.323342921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:08:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:08:01.322180843 +0000 UTC m=+4825.117529409" watchObservedRunningTime="2026-11-02 04:08:01.323342921 +0000 UTC m=+4825.118691506"
Nov 02 04:08:02.214402 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:02.214369    4407 generic.go:356] "Generic (PLEG): container finished" podID="389d6a5d-c283-47de-8286-a82d32bab4d6" containerID="c8420820f044f572ece10074d76e59060ed1549c930fdb967aaa7acc2b110bb7" exitCode=0
Nov 02 04:08:02.214810 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:02.214456    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw" event={"ID":"389d6a5d-c283-47de-8286-a82d32bab4d6","Type":"ContainerDied","Data":"c8420820f044f572ece10074d76e59060ed1549c930fdb967aaa7acc2b110bb7"}
Nov 02 04:08:02.700187 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:02.700148    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:03.355716 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.355694    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:03.424252 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.424225    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-xks-ca\") pod \"389d6a5d-c283-47de-8286-a82d32bab4d6\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") "
Nov 02 04:08:03.424421 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.424313    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-openshift-ca\") pod \"389d6a5d-c283-47de-8286-a82d32bab4d6\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") "
Nov 02 04:08:03.424421 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.424348    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/389d6a5d-c283-47de-8286-a82d32bab4d6-kube-api-access-25778\" (UniqueName: \"kubernetes.io/projected/389d6a5d-c283-47de-8286-a82d32bab4d6-kube-api-access-25778\") pod \"389d6a5d-c283-47de-8286-a82d32bab4d6\" (UID: \"389d6a5d-c283-47de-8286-a82d32bab4d6\") "
Nov 02 04:08:03.424710 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.424686    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-openshift-ca" pod "389d6a5d-c283-47de-8286-a82d32bab4d6" (UID: "389d6a5d-c283-47de-8286-a82d32bab4d6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:08:03.426383 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.426362    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-xks-ca" pod "389d6a5d-c283-47de-8286-a82d32bab4d6" (UID: "389d6a5d-c283-47de-8286-a82d32bab4d6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:08:03.426484 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.426464    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/389d6a5d-c283-47de-8286-a82d32bab4d6-kube-api-access-25778" pod "389d6a5d-c283-47de-8286-a82d32bab4d6" (UID: "389d6a5d-c283-47de-8286-a82d32bab4d6"). InnerVolumeSpecName "kube-api-access-25778". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:08:03.525990 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.525959    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-xks-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:03.525990 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.525987    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/389d6a5d-c283-47de-8286-a82d32bab4d6-maas-api-openshift-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:03.525990 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:03.525998    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25778\" (UniqueName: \"kubernetes.io/projected/389d6a5d-c283-47de-8286-a82d32bab4d6-kube-api-access-25778\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:04.225075 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:04.225039    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw" event={"ID":"389d6a5d-c283-47de-8286-a82d32bab4d6","Type":"ContainerDied","Data":"ae3b3e44849740ed49c84741b2124bcd95b6e47dd2a19bca35a10fb620b26b3f"}
Nov 02 04:08:04.225075 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:04.225061    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"
Nov 02 04:08:04.225350 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:04.225088    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae3b3e44849740ed49c84741b2124bcd95b6e47dd2a19bca35a10fb620b26b3f"
Nov 02 04:08:05.187764 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:08:05.187728    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:08:07.577135 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:07.577104    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:11.545064 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:11.545029    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"]
Nov 02 04:08:11.651780 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:11.651748    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"]
Nov 02 04:08:11.929581 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:11.929496    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:12.093013 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:12.092968    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:12.356114 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:12.356066    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:16.546159 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:16.546120    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt" podUID="7029ba4f-93aa-4d8c-aba4-b1fdaf275526" containerName="payload-processing" containerID="cri-o://b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3" gracePeriod=25
Nov 02 04:08:16.653093 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:16.653032    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx" podUID="0f37870b-0726-49e6-92ea-a55914f57f2b" containerName="payload-pre-processing" containerID="cri-o://f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1" gracePeriod=25
Nov 02 04:08:16.799656 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:16.799588    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:08:16.922486 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:16.922453    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:08:16.960288 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:16.960262    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-kube-api-access-v8gp4\" (UniqueName: \"kubernetes.io/projected/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-kube-api-access-v8gp4\") pod \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\" (UID: \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\") "
Nov 02 04:08:16.960472 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:16.960329    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-plugins-config-volume\") pod \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\" (UID: \"7029ba4f-93aa-4d8c-aba4-b1fdaf275526\") "
Nov 02 04:08:16.960824 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:16.960799    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-plugins-config-volume" pod "7029ba4f-93aa-4d8c-aba4-b1fdaf275526" (UID: "7029ba4f-93aa-4d8c-aba4-b1fdaf275526"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:08:16.962565 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:16.962532    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-kube-api-access-v8gp4" pod "7029ba4f-93aa-4d8c-aba4-b1fdaf275526" (UID: "7029ba4f-93aa-4d8c-aba4-b1fdaf275526"). InnerVolumeSpecName "kube-api-access-v8gp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:08:17.061811 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.061724    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f37870b-0726-49e6-92ea-a55914f57f2b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f37870b-0726-49e6-92ea-a55914f57f2b-plugins-config-volume\") pod \"0f37870b-0726-49e6-92ea-a55914f57f2b\" (UID: \"0f37870b-0726-49e6-92ea-a55914f57f2b\") "
Nov 02 04:08:17.061999 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.061827    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f37870b-0726-49e6-92ea-a55914f57f2b-kube-api-access-kj56t\" (UniqueName: \"kubernetes.io/projected/0f37870b-0726-49e6-92ea-a55914f57f2b-kube-api-access-kj56t\") pod \"0f37870b-0726-49e6-92ea-a55914f57f2b\" (UID: \"0f37870b-0726-49e6-92ea-a55914f57f2b\") "
Nov 02 04:08:17.062181 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.062145    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f37870b-0726-49e6-92ea-a55914f57f2b-plugins-config-volume" pod "0f37870b-0726-49e6-92ea-a55914f57f2b" (UID: "0f37870b-0726-49e6-92ea-a55914f57f2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:08:17.062292 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.062204    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8gp4\" (UniqueName: \"kubernetes.io/projected/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-kube-api-access-v8gp4\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:17.062292 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.062217    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7029ba4f-93aa-4d8c-aba4-b1fdaf275526-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:17.064097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.064070    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f37870b-0726-49e6-92ea-a55914f57f2b-kube-api-access-kj56t" pod "0f37870b-0726-49e6-92ea-a55914f57f2b" (UID: "0f37870b-0726-49e6-92ea-a55914f57f2b"). InnerVolumeSpecName "kube-api-access-kj56t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:08:17.129959 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.129921    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:17.163224 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.163193    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj56t\" (UniqueName: \"kubernetes.io/projected/0f37870b-0726-49e6-92ea-a55914f57f2b-kube-api-access-kj56t\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:17.163224 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.163223    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f37870b-0726-49e6-92ea-a55914f57f2b-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:17.287669 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.287617    4407 generic.go:356] "Generic (PLEG): container finished" podID="7029ba4f-93aa-4d8c-aba4-b1fdaf275526" containerID="b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3" exitCode=0
Nov 02 04:08:17.287862 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.287683    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"
Nov 02 04:08:17.287862 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.287719    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt" event={"ID":"7029ba4f-93aa-4d8c-aba4-b1fdaf275526","Type":"ContainerDied","Data":"b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3"}
Nov 02 04:08:17.287862 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.287781    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt" event={"ID":"7029ba4f-93aa-4d8c-aba4-b1fdaf275526","Type":"ContainerDied","Data":"7407ae2e26aa0232fe64b15c3930ffe8a0b559a5b45239fb91756b451553b049"}
Nov 02 04:08:17.287862 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.287806    4407 scope.go:122] "RemoveContainer" containerID="b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3"
Nov 02 04:08:17.289205 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.289183    4407 generic.go:356] "Generic (PLEG): container finished" podID="0f37870b-0726-49e6-92ea-a55914f57f2b" containerID="f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1" exitCode=0
Nov 02 04:08:17.289282 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.289235    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"
Nov 02 04:08:17.289282 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.289256    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx" event={"ID":"0f37870b-0726-49e6-92ea-a55914f57f2b","Type":"ContainerDied","Data":"f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1"}
Nov 02 04:08:17.289366 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.289296    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx" event={"ID":"0f37870b-0726-49e6-92ea-a55914f57f2b","Type":"ContainerDied","Data":"3aa7af85b841c1624cff69829e824e960c5c7751e04b61acec4f5d556f52d5af"}
Nov 02 04:08:17.297068 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.297050    4407 scope.go:122] "RemoveContainer" containerID="b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3"
Nov 02 04:08:17.297377 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:08:17.297349    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3\": container with ID starting with b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3 not found: ID does not exist" containerID="b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3"
Nov 02 04:08:17.297497 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.297388    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3"} err="failed to get container status \"b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3\": rpc error: code = NotFound desc = could not find container \"b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3\": container with ID starting with b125286311461317ac31b0a276d6a85e5a0d1d516d869b22f978fb4a03c4d0a3 not found: ID does not exist"
Nov 02 04:08:17.297497 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.297410    4407 scope.go:122] "RemoveContainer" containerID="f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1"
Nov 02 04:08:17.305877 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.305861    4407 scope.go:122] "RemoveContainer" containerID="f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1"
Nov 02 04:08:17.306138 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:08:17.306116    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1\": container with ID starting with f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1 not found: ID does not exist" containerID="f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1"
Nov 02 04:08:17.306206 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.306144    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1"} err="failed to get container status \"f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1\": rpc error: code = NotFound desc = could not find container \"f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1\": container with ID starting with f92167850a3ae1293e06c84d87037113c6951a16a3d53ee2b11e4676813344e1 not found: ID does not exist"
Nov 02 04:08:17.453745 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.453667    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"]
Nov 02 04:08:17.504510 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.504481    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-12c92e-794c95d6c5-xqkjx"]
Nov 02 04:08:17.568653 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.568609    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"]
Nov 02 04:08:17.570341 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:17.570319    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-12c92e-77cc49c889-lrmqt"]
Nov 02 04:08:18.357088 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:18.357052    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f37870b-0726-49e6-92ea-a55914f57f2b" path="/var/lib/kubelet/pods/0f37870b-0726-49e6-92ea-a55914f57f2b/volumes"
Nov 02 04:08:18.357405 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:18.357393    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7029ba4f-93aa-4d8c-aba4-b1fdaf275526" path="/var/lib/kubelet/pods/7029ba4f-93aa-4d8c-aba4-b1fdaf275526/volumes"
Nov 02 04:08:21.972381 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:21.972342    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:26.319777 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:26.319743    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:27.746883 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:27.746848    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:27.981753 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:27.981722    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:28.168914 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:28.168828    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:28.232489 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:28.232453    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"]
Nov 02 04:08:28.282176 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:28.282143    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-12c92e-kb8sw"]
Nov 02 04:08:28.350605 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:28.350564    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="389d6a5d-c283-47de-8286-a82d32bab4d6" path="/var/lib/kubelet/pods/389d6a5d-c283-47de-8286-a82d32bab4d6/volumes"
Nov 02 04:08:28.643940 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:28.643894    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:31.776505 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:31.776471    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:08:33.122134 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:33.122102    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:34.212196 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:34.212158    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:34.382861 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:34.382823    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:34.487985 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:34.487897    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"]
Nov 02 04:08:34.488165 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:34.488146    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy" containerID="cri-o://ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f" gracePeriod=30
Nov 02 04:08:34.566434 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:34.566400    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:35.189023 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:08:35.188990    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:08:35.307911 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.307878    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"]
Nov 02 04:08:35.308300 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308265    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0f37870b-0726-49e6-92ea-a55914f57f2b" containerName="payload-pre-processing"
Nov 02 04:08:35.308300 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308277    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f37870b-0726-49e6-92ea-a55914f57f2b" containerName="payload-pre-processing"
Nov 02 04:08:35.308300 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308288    4407 cpu_manager.go:414] "containerMap: removing container" podUID="7029ba4f-93aa-4d8c-aba4-b1fdaf275526" containerName="payload-processing"
Nov 02 04:08:35.308300 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308293    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7029ba4f-93aa-4d8c-aba4-b1fdaf275526" containerName="payload-processing"
Nov 02 04:08:35.308431 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308316    4407 cpu_manager.go:414] "containerMap: removing container" podUID="389d6a5d-c283-47de-8286-a82d32bab4d6" containerName="revoke-keys"
Nov 02 04:08:35.308431 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308321    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389d6a5d-c283-47de-8286-a82d32bab4d6" containerName="revoke-keys"
Nov 02 04:08:35.308431 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308390    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="389d6a5d-c283-47de-8286-a82d32bab4d6" containerName="revoke-keys"
Nov 02 04:08:35.308431 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308403    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f37870b-0726-49e6-92ea-a55914f57f2b" containerName="payload-pre-processing"
Nov 02 04:08:35.308431 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.308409    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="7029ba4f-93aa-4d8c-aba4-b1fdaf275526" containerName="payload-processing"
Nov 02 04:08:35.311788 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.311773    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.341856 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.341833    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss2p5\" (UniqueName: \"kubernetes.io/projected/a463137b-485a-452c-8398-9ef8847361a5-kube-api-access-ss2p5\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.341997 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.341897    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a463137b-485a-452c-8398-9ef8847361a5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.342041 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.342013    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a463137b-485a-452c-8398-9ef8847361a5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.368711 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.368691    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 04:08:35.368863 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.368782    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9z5bk\""
Nov 02 04:08:35.371612 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.371594    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"]
Nov 02 04:08:35.443326 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.443236    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a463137b-485a-452c-8398-9ef8847361a5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.443690 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.443663    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss2p5\" (UniqueName: \"kubernetes.io/projected/a463137b-485a-452c-8398-9ef8847361a5-kube-api-access-ss2p5\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.443863 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.443844    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a463137b-485a-452c-8398-9ef8847361a5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.444587 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.444564    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a463137b-485a-452c-8398-9ef8847361a5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.446116 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.446086    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a463137b-485a-452c-8398-9ef8847361a5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.501880 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.501847    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss2p5\" (UniqueName: \"kubernetes.io/projected/a463137b-485a-452c-8398-9ef8847361a5-kube-api-access-ss2p5\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.624300 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.624266    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:35.994741 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:35.994716    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"]
Nov 02 04:08:35.996195 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:08:35.996168    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda463137b_485a_452c_8398_9ef8847361a5.slice/crio-000e48936a494f98f4d561d5a3fe20b9eacca3dd4ae446945bcfac10c7deb2e7 WatchSource:0}: Error finding container 000e48936a494f98f4d561d5a3fe20b9eacca3dd4ae446945bcfac10c7deb2e7: Status 404 returned error can't find the container with id 000e48936a494f98f4d561d5a3fe20b9eacca3dd4ae446945bcfac10c7deb2e7
Nov 02 04:08:36.389679 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:36.389644    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n" event={"ID":"a463137b-485a-452c-8398-9ef8847361a5","Type":"ContainerStarted","Data":"c39aafc323616c9deead3005cc73a8c2d45de2e6b7741fb7659022384aac30e4"}
Nov 02 04:08:36.390092 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:36.389700    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n" event={"ID":"a463137b-485a-452c-8398-9ef8847361a5","Type":"ContainerStarted","Data":"000e48936a494f98f4d561d5a3fe20b9eacca3dd4ae446945bcfac10c7deb2e7"}
Nov 02 04:08:36.505876 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:36.505827    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n" podStartSLOduration=1.505812041 podStartE2EDuration="1.505812041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:08:36.504845266 +0000 UTC m=+4860.300193833" watchObservedRunningTime="2026-11-02 04:08:36.505812041 +0000 UTC m=+4860.301160605"
Nov 02 04:08:37.395653 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:37.395591    4407 generic.go:356] "Generic (PLEG): container finished" podID="a463137b-485a-452c-8398-9ef8847361a5" containerID="c39aafc323616c9deead3005cc73a8c2d45de2e6b7741fb7659022384aac30e4" exitCode=0
Nov 02 04:08:37.396094 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:37.395666    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n" event={"ID":"a463137b-485a-452c-8398-9ef8847361a5","Type":"ContainerDied","Data":"c39aafc323616c9deead3005cc73a8c2d45de2e6b7741fb7659022384aac30e4"}
Nov 02 04:08:37.994559 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:37.994528    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:38.539875 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.539852    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:38.577923 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.577894    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a463137b-485a-452c-8398-9ef8847361a5-kube-api-access-ss2p5\" (UniqueName: \"kubernetes.io/projected/a463137b-485a-452c-8398-9ef8847361a5-kube-api-access-ss2p5\") pod \"a463137b-485a-452c-8398-9ef8847361a5\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") "
Nov 02 04:08:38.578067 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.577974    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a463137b-485a-452c-8398-9ef8847361a5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a463137b-485a-452c-8398-9ef8847361a5-maas-api-xks-ca\") pod \"a463137b-485a-452c-8398-9ef8847361a5\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") "
Nov 02 04:08:38.578067 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.578011    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a463137b-485a-452c-8398-9ef8847361a5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a463137b-485a-452c-8398-9ef8847361a5-maas-api-openshift-ca\") pod \"a463137b-485a-452c-8398-9ef8847361a5\" (UID: \"a463137b-485a-452c-8398-9ef8847361a5\") "
Nov 02 04:08:38.578426 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.578400    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a463137b-485a-452c-8398-9ef8847361a5-maas-api-openshift-ca" pod "a463137b-485a-452c-8398-9ef8847361a5" (UID: "a463137b-485a-452c-8398-9ef8847361a5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:08:38.580098 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.580069    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a463137b-485a-452c-8398-9ef8847361a5-kube-api-access-ss2p5" pod "a463137b-485a-452c-8398-9ef8847361a5" (UID: "a463137b-485a-452c-8398-9ef8847361a5"). InnerVolumeSpecName "kube-api-access-ss2p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:08:38.580187 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.580116    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a463137b-485a-452c-8398-9ef8847361a5-maas-api-xks-ca" pod "a463137b-485a-452c-8398-9ef8847361a5" (UID: "a463137b-485a-452c-8398-9ef8847361a5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:08:38.679199 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.679120    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss2p5\" (UniqueName: \"kubernetes.io/projected/a463137b-485a-452c-8398-9ef8847361a5-kube-api-access-ss2p5\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:38.679199 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.679149    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a463137b-485a-452c-8398-9ef8847361a5-maas-api-xks-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:38.679199 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:38.679159    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a463137b-485a-452c-8398-9ef8847361a5-maas-api-openshift-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.406263 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.406229    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n" event={"ID":"a463137b-485a-452c-8398-9ef8847361a5","Type":"ContainerDied","Data":"000e48936a494f98f4d561d5a3fe20b9eacca3dd4ae446945bcfac10c7deb2e7"}
Nov 02 04:08:39.406421 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.406279    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="000e48936a494f98f4d561d5a3fe20b9eacca3dd4ae446945bcfac10c7deb2e7"
Nov 02 04:08:39.406421 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.406244    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"
Nov 02 04:08:39.742700 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.742672    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:08:39.790971 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.790939    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-token\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-token\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791119 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.790999    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-kube-api-access-5czjw\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-kube-api-access-5czjw\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791119 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791033    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-data\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791119 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791069    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdf2cf02-113e-4472-a59d-30abda30c9d3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdf2cf02-113e-4472-a59d-30abda30c9d3-istiod-ca-cert\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791241 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791124    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-certs\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791241 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791162    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-podinfo\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791345 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791321    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-credential-socket\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791384 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791351    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-data" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:08:39.791438 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791413    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-socket\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791494 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791455    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-envoy\") pod \"bdf2cf02-113e-4472-a59d-30abda30c9d3\" (UID: \"bdf2cf02-113e-4472-a59d-30abda30c9d3\") "
Nov 02 04:08:39.791607 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791569    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-certs" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:08:39.791734 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791660    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-credential-socket" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:08:39.791734 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791676    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-socket" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:08:39.791734 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791717    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdf2cf02-113e-4472-a59d-30abda30c9d3-istiod-ca-cert" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:08:39.791912 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791859    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.791912 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791877    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.791912 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791887    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.791912 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791900    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdf2cf02-113e-4472-a59d-30abda30c9d3-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.792080 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.791918    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.793461 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.793438    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-podinfo" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:08:39.793768 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.793748    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-envoy" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:08:39.794022 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.794001    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-token" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:08:39.794078 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.794034    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-kube-api-access-5czjw" pod "bdf2cf02-113e-4472-a59d-30abda30c9d3" (UID: "bdf2cf02-113e-4472-a59d-30abda30c9d3"). InnerVolumeSpecName "kube-api-access-5czjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:08:39.892612 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.892581    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5czjw\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-kube-api-access-5czjw\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.892612 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.892606    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.892612 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.892615    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:39.892878 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:39.892643    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdf2cf02-113e-4472-a59d-30abda30c9d3-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:08:40.412207 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.412170    4407 generic.go:356] "Generic (PLEG): container finished" podID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerID="ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f" exitCode=0
Nov 02 04:08:40.412364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.412225    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"
Nov 02 04:08:40.412364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.412249    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" event={"ID":"bdf2cf02-113e-4472-a59d-30abda30c9d3","Type":"ContainerDied","Data":"ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f"}
Nov 02 04:08:40.412364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.412312    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j" event={"ID":"bdf2cf02-113e-4472-a59d-30abda30c9d3","Type":"ContainerDied","Data":"75d6be05ac6974e6394a55a5846ccb1b79f0e608ad594e5050ba0347cbc454a5"}
Nov 02 04:08:40.412364 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.412338    4407 scope.go:122] "RemoveContainer" containerID="ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f"
Nov 02 04:08:40.421369 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.421350    4407 scope.go:122] "RemoveContainer" containerID="4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c"
Nov 02 04:08:40.429436 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.429400    4407 scope.go:122] "RemoveContainer" containerID="ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f"
Nov 02 04:08:40.429751 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:08:40.429731    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f\": container with ID starting with ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f not found: ID does not exist" containerID="ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f"
Nov 02 04:08:40.429808 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.429761    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f"} err="failed to get container status \"ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f\": rpc error: code = NotFound desc = could not find container \"ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f\": container with ID starting with ac1dc56541401d88db5da2217114c2b1580bfbac26449a9cfb0dda6769eabe9f not found: ID does not exist"
Nov 02 04:08:40.429808 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.429779    4407 scope.go:122] "RemoveContainer" containerID="4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c"
Nov 02 04:08:40.430024 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:08:40.430006    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c\": container with ID starting with 4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c not found: ID does not exist" containerID="4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c"
Nov 02 04:08:40.430069 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.430032    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c"} err="failed to get container status \"4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c\": rpc error: code = NotFound desc = could not find container \"4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c\": container with ID starting with 4490fa6a32346328850a39da7ee72f822c1a8715cbb9d846e33c30745494213c not found: ID does not exist"
Nov 02 04:08:40.579999 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.579964    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"]
Nov 02 04:08:40.630883 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:40.630807    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-12c92e-openshift-default-6df5f5d7df-frm7j"]
Nov 02 04:08:42.349909 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:42.349878    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" path="/var/lib/kubelet/pods/bdf2cf02-113e-4472-a59d-30abda30c9d3/volumes"
Nov 02 04:08:43.071797 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:43.071755    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:47.169666 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:47.169607    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:47.351843 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:47.351808    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:47.887669 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:47.887602    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:52.682562 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:52.682520    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:55.867498 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:55.867450    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:56.825975 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:56.825914    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:58.347618 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:58.347575    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:58.573873 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:58.573839    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:58.732936 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:58.732847    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:08:58.828890 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:58.828858    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"]
Nov 02 04:08:58.879231 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:58.879202    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-aad578-j6b4n"]
Nov 02 04:08:59.255567 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:08:59.255528    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:00.350196 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:00.350162    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a463137b-485a-452c-8398-9ef8847361a5" path="/var/lib/kubelet/pods/a463137b-485a-452c-8398-9ef8847361a5/volumes"
Nov 02 04:09:02.364180 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:02.364145    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:09:03.690120 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:03.690090    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:05.120833 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:05.120799    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:05.187476 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:09:05.187431    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:09:05.288716 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:05.288684    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:05.460531 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:05.460442    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:06.278167 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.278134    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"]
Nov 02 04:09:06.278543 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.278513    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy"
Nov 02 04:09:06.278543 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.278524    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy"
Nov 02 04:09:06.278653 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.278548    4407 cpu_manager.go:414] "containerMap: removing container" podUID="a463137b-485a-452c-8398-9ef8847361a5" containerName="revoke-keys"
Nov 02 04:09:06.278653 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.278553    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a463137b-485a-452c-8398-9ef8847361a5" containerName="revoke-keys"
Nov 02 04:09:06.278653 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.278632    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy"
Nov 02 04:09:06.278653 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.278644    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="a463137b-485a-452c-8398-9ef8847361a5" containerName="revoke-keys"
Nov 02 04:09:06.278653 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.278652    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy"
Nov 02 04:09:06.281934 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.281919    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.338677 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.338647    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"]
Nov 02 04:09:06.355089 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.355062    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.355272 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.355097    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkfr4\" (UniqueName: \"kubernetes.io/projected/0a559095-4896-4520-8bdc-55c7d4f34be5-kube-api-access-vkfr4\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.355272 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.355117    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.386545 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.386517    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 04:09:06.386717 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.386570    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9z5bk\""
Nov 02 04:09:06.456208 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.456179    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.456411 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.456217    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkfr4\" (UniqueName: \"kubernetes.io/projected/0a559095-4896-4520-8bdc-55c7d4f34be5-kube-api-access-vkfr4\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.456411 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.456236    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.456882 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.456863    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.458757 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.458734    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.513512 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.513465    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkfr4\" (UniqueName: \"kubernetes.io/projected/0a559095-4896-4520-8bdc-55c7d4f34be5-kube-api-access-vkfr4\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.594208 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.594120    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:06.965164 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:06.965139    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"]
Nov 02 04:09:06.967782 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:09:06.967759    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a559095_4896_4520_8bdc_55c7d4f34be5.slice/crio-fe0da86a5e21416f43288020d63e0d9625fda9918751a136a0be1614b8fa9891 WatchSource:0}: Error finding container fe0da86a5e21416f43288020d63e0d9625fda9918751a136a0be1614b8fa9891: Status 404 returned error can't find the container with id fe0da86a5e21416f43288020d63e0d9625fda9918751a136a0be1614b8fa9891
Nov 02 04:09:07.540845 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:07.540751    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj" event={"ID":"0a559095-4896-4520-8bdc-55c7d4f34be5","Type":"ContainerStarted","Data":"0d6d8c3468ade9692dbf3baf1ea43cdeead91dfc51a74203cefff99ce3308698"}
Nov 02 04:09:07.540845 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:07.540809    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj" event={"ID":"0a559095-4896-4520-8bdc-55c7d4f34be5","Type":"ContainerStarted","Data":"fe0da86a5e21416f43288020d63e0d9625fda9918751a136a0be1614b8fa9891"}
Nov 02 04:09:07.656741 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:07.656687    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj" podStartSLOduration=1.6566736450000001 podStartE2EDuration="1.656673645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:09:07.654767496 +0000 UTC m=+4891.450116060" watchObservedRunningTime="2026-11-02 04:09:07.656673645 +0000 UTC m=+4891.452022210"
Nov 02 04:09:08.547053 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:08.547016    4407 generic.go:356] "Generic (PLEG): container finished" podID="0a559095-4896-4520-8bdc-55c7d4f34be5" containerID="0d6d8c3468ade9692dbf3baf1ea43cdeead91dfc51a74203cefff99ce3308698" exitCode=0
Nov 02 04:09:08.547470 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:08.547099    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj" event={"ID":"0a559095-4896-4520-8bdc-55c7d4f34be5","Type":"ContainerDied","Data":"0d6d8c3468ade9692dbf3baf1ea43cdeead91dfc51a74203cefff99ce3308698"}
Nov 02 04:09:08.791061 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:08.791013    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:09.682328 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.682304    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:09.787893 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.787862    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-openshift-ca\") pod \"0a559095-4896-4520-8bdc-55c7d4f34be5\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") "
Nov 02 04:09:09.788083 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.787981    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-xks-ca\") pod \"0a559095-4896-4520-8bdc-55c7d4f34be5\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") "
Nov 02 04:09:09.788083 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.788007    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a559095-4896-4520-8bdc-55c7d4f34be5-kube-api-access-vkfr4\" (UniqueName: \"kubernetes.io/projected/0a559095-4896-4520-8bdc-55c7d4f34be5-kube-api-access-vkfr4\") pod \"0a559095-4896-4520-8bdc-55c7d4f34be5\" (UID: \"0a559095-4896-4520-8bdc-55c7d4f34be5\") "
Nov 02 04:09:09.788221 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.788199    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-openshift-ca" pod "0a559095-4896-4520-8bdc-55c7d4f34be5" (UID: "0a559095-4896-4520-8bdc-55c7d4f34be5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:09:09.790220 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.790199    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-xks-ca" pod "0a559095-4896-4520-8bdc-55c7d4f34be5" (UID: "0a559095-4896-4520-8bdc-55c7d4f34be5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:09:09.790332 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.790253    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a559095-4896-4520-8bdc-55c7d4f34be5-kube-api-access-vkfr4" pod "0a559095-4896-4520-8bdc-55c7d4f34be5" (UID: "0a559095-4896-4520-8bdc-55c7d4f34be5"). InnerVolumeSpecName "kube-api-access-vkfr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:09:09.889215 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.889144    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-openshift-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:09:09.889215 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.889171    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a559095-4896-4520-8bdc-55c7d4f34be5-maas-api-xks-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:09:09.889215 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:09.889181    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkfr4\" (UniqueName: \"kubernetes.io/projected/0a559095-4896-4520-8bdc-55c7d4f34be5-kube-api-access-vkfr4\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:09:10.561724 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:10.561694    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj" event={"ID":"0a559095-4896-4520-8bdc-55c7d4f34be5","Type":"ContainerDied","Data":"fe0da86a5e21416f43288020d63e0d9625fda9918751a136a0be1614b8fa9891"}
Nov 02 04:09:10.561895 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:10.561741    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe0da86a5e21416f43288020d63e0d9625fda9918751a136a0be1614b8fa9891"
Nov 02 04:09:10.561895 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:10.561705    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"
Nov 02 04:09:13.778224 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:13.778193    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:17.877889 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:17.877855    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"]
Nov 02 04:09:18.038807 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:18.038738    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:18.198453 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:18.198365    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:18.349710 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:18.349682    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:19.861114 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:19.861083    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:22.133815 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:22.133776    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:22.426865 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:22.426726    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:22.665986 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:22.665948    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"]
Nov 02 04:09:22.716959 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:22.716871    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3d14ee-4njpj"]
Nov 02 04:09:22.878271 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:22.878232    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd" podUID="eed0c132-c992-4390-923a-5cbeadffa424" containerName="payload-pre-processing" containerID="cri-o://4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409" gracePeriod=25
Nov 02 04:09:23.063070 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:23.063035    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:23.406552 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:23.406478    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:09:24.350814 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:24.350783    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a559095-4896-4520-8bdc-55c7d4f34be5" path="/var/lib/kubelet/pods/0a559095-4896-4520-8bdc-55c7d4f34be5/volumes"
Nov 02 04:09:24.800321 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:24.800286    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:27.297949 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:27.297912    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd" podUID="eed0c132-c992-4390-923a-5cbeadffa424" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:09:28.943392 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:28.943331    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:29.118119 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:29.118090    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:29.303289 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:29.303251    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:29.737381 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:29.737308    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:29.902731 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:29.902684    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:34.853482 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:34.853448    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:35.190047 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:09:35.189949    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:09:35.208577 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:35.208547    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:37.297195 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:37.297160    4407 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd" podUID="eed0c132-c992-4390-923a-5cbeadffa424" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 04:09:40.078368 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:40.078335    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:42.094955 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:42.094918    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:42.254845 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:42.254816    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:43.123189 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.123156    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 04:09:43.220749 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.220663    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eed0c132-c992-4390-923a-5cbeadffa424-kube-api-access-qn4gh\" (UniqueName: \"kubernetes.io/projected/eed0c132-c992-4390-923a-5cbeadffa424-kube-api-access-qn4gh\") pod \"eed0c132-c992-4390-923a-5cbeadffa424\" (UID: \"eed0c132-c992-4390-923a-5cbeadffa424\") "
Nov 02 04:09:43.220749 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.220728    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eed0c132-c992-4390-923a-5cbeadffa424-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed0c132-c992-4390-923a-5cbeadffa424-plugins-config-volume\") pod \"eed0c132-c992-4390-923a-5cbeadffa424\" (UID: \"eed0c132-c992-4390-923a-5cbeadffa424\") "
Nov 02 04:09:43.221094 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.221071    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eed0c132-c992-4390-923a-5cbeadffa424-plugins-config-volume" pod "eed0c132-c992-4390-923a-5cbeadffa424" (UID: "eed0c132-c992-4390-923a-5cbeadffa424"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:09:43.222910 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.222883    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eed0c132-c992-4390-923a-5cbeadffa424-kube-api-access-qn4gh" pod "eed0c132-c992-4390-923a-5cbeadffa424" (UID: "eed0c132-c992-4390-923a-5cbeadffa424"). InnerVolumeSpecName "kube-api-access-qn4gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:09:43.322011 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.321936    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn4gh\" (UniqueName: \"kubernetes.io/projected/eed0c132-c992-4390-923a-5cbeadffa424-kube-api-access-qn4gh\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:09:43.322011 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.321975    4407 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed0c132-c992-4390-923a-5cbeadffa424-plugins-config-volume\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:09:43.720924 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.720891    4407 generic.go:356] "Generic (PLEG): container finished" podID="eed0c132-c992-4390-923a-5cbeadffa424" containerID="4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409" exitCode=0
Nov 02 04:09:43.721146 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.720942    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"
Nov 02 04:09:43.721146 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.720980    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd" event={"ID":"eed0c132-c992-4390-923a-5cbeadffa424","Type":"ContainerDied","Data":"4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409"}
Nov 02 04:09:43.721146 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.721042    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd" event={"ID":"eed0c132-c992-4390-923a-5cbeadffa424","Type":"ContainerDied","Data":"f32b86f799b17023a1e4fef1cd8e96f05aa8d57088724a8c8ec0928d34c7c30d"}
Nov 02 04:09:43.721146 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.721070    4407 scope.go:122] "RemoveContainer" containerID="4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409"
Nov 02 04:09:43.732581 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.732564    4407 scope.go:122] "RemoveContainer" containerID="4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409"
Nov 02 04:09:43.732881 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:09:43.732861    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409\": container with ID starting with 4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409 not found: ID does not exist" containerID="4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409"
Nov 02 04:09:43.732945 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.732890    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409"} err="failed to get container status \"4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409\": rpc error: code = NotFound desc = could not find container \"4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409\": container with ID starting with 4189d685e8092df59e02cbd623077fcf7c9d29e1692d130ab6a7b88773960409 not found: ID does not exist"
Nov 02 04:09:43.887270 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.887231    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"]
Nov 02 04:09:43.911871 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:43.911840    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3d14ee-b7dd4c6b7-c4wvd"]
Nov 02 04:09:44.350475 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:44.350433    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eed0c132-c992-4390-923a-5cbeadffa424" path="/var/lib/kubelet/pods/eed0c132-c992-4390-923a-5cbeadffa424/volumes"
Nov 02 04:09:44.775827 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:44.775788    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:09:49.059839 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.059806    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"]
Nov 02 04:09:49.060355 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.060234    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy"
Nov 02 04:09:49.060355 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.060246    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdf2cf02-113e-4472-a59d-30abda30c9d3" containerName="istio-proxy"
Nov 02 04:09:49.060355 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.060258    4407 cpu_manager.go:414] "containerMap: removing container" podUID="0a559095-4896-4520-8bdc-55c7d4f34be5" containerName="revoke-keys"
Nov 02 04:09:49.060355 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.060264    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a559095-4896-4520-8bdc-55c7d4f34be5" containerName="revoke-keys"
Nov 02 04:09:49.060355 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.060277    4407 cpu_manager.go:414] "containerMap: removing container" podUID="eed0c132-c992-4390-923a-5cbeadffa424" containerName="payload-pre-processing"
Nov 02 04:09:49.060355 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.060282    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eed0c132-c992-4390-923a-5cbeadffa424" containerName="payload-pre-processing"
Nov 02 04:09:49.060704 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.060373    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a559095-4896-4520-8bdc-55c7d4f34be5" containerName="revoke-keys"
Nov 02 04:09:49.060704 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.060386    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="eed0c132-c992-4390-923a-5cbeadffa424" containerName="payload-pre-processing"
Nov 02 04:09:49.064898 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.064882    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.119739 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.119704    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"]
Nov 02 04:09:49.167934 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.167894    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 04:09:49.168131 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.168112    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9z5bk\""
Nov 02 04:09:49.174525 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.174499    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.174673 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.174562    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.174727 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.174688    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2xgz\" (UniqueName: \"kubernetes.io/projected/bfb08982-dbce-4bb0-9149-9f31f69a9911-kube-api-access-k2xgz\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.276118 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.276083    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.276350 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.276149    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2xgz\" (UniqueName: \"kubernetes.io/projected/bfb08982-dbce-4bb0-9149-9f31f69a9911-kube-api-access-k2xgz\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.276350 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.276218    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.276761 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.276741    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.278695 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.278674    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.333788 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.333714    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2xgz\" (UniqueName: \"kubernetes.io/projected/bfb08982-dbce-4bb0-9149-9f31f69a9911-kube-api-access-k2xgz\") pod \"maas-api-revoke-sub-ae3e5146-bw7sw\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.377044 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.377017    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:49.747260 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:09:49.747222    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb08982_dbce_4bb0_9149_9f31f69a9911.slice/crio-a8f00308b491c974b98e352066fec60466874977011dc12ca53de9e533ce677d WatchSource:0}: Error finding container a8f00308b491c974b98e352066fec60466874977011dc12ca53de9e533ce677d: Status 404 returned error can't find the container with id a8f00308b491c974b98e352066fec60466874977011dc12ca53de9e533ce677d
Nov 02 04:09:49.749132 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:49.749106    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"]
Nov 02 04:09:50.756063 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:50.756028    4407 generic.go:356] "Generic (PLEG): container finished" podID="bfb08982-dbce-4bb0-9149-9f31f69a9911" containerID="f00fea30c652f68ac79a65eaf179af7026f45e2671e4964d6111213faa9ab64b" exitCode=0
Nov 02 04:09:50.756470 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:50.756085    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw" event={"ID":"bfb08982-dbce-4bb0-9149-9f31f69a9911","Type":"ContainerDied","Data":"f00fea30c652f68ac79a65eaf179af7026f45e2671e4964d6111213faa9ab64b"}
Nov 02 04:09:50.756470 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:50.756121    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw" event={"ID":"bfb08982-dbce-4bb0-9149-9f31f69a9911","Type":"ContainerStarted","Data":"a8f00308b491c974b98e352066fec60466874977011dc12ca53de9e533ce677d"}
Nov 02 04:09:51.905284 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:51.905260    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:52.004390 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.004355    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfb08982-dbce-4bb0-9149-9f31f69a9911-kube-api-access-k2xgz\" (UniqueName: \"kubernetes.io/projected/bfb08982-dbce-4bb0-9149-9f31f69a9911-kube-api-access-k2xgz\") pod \"bfb08982-dbce-4bb0-9149-9f31f69a9911\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") "
Nov 02 04:09:52.004591 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.004485    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-openshift-ca\") pod \"bfb08982-dbce-4bb0-9149-9f31f69a9911\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") "
Nov 02 04:09:52.004591 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.004521    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-xks-ca\") pod \"bfb08982-dbce-4bb0-9149-9f31f69a9911\" (UID: \"bfb08982-dbce-4bb0-9149-9f31f69a9911\") "
Nov 02 04:09:52.004846 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.004820    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-openshift-ca" pod "bfb08982-dbce-4bb0-9149-9f31f69a9911" (UID: "bfb08982-dbce-4bb0-9149-9f31f69a9911"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:09:52.006698 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.006673    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-xks-ca" pod "bfb08982-dbce-4bb0-9149-9f31f69a9911" (UID: "bfb08982-dbce-4bb0-9149-9f31f69a9911"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 04:09:52.006794 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.006746    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb08982-dbce-4bb0-9149-9f31f69a9911-kube-api-access-k2xgz" pod "bfb08982-dbce-4bb0-9149-9f31f69a9911" (UID: "bfb08982-dbce-4bb0-9149-9f31f69a9911"). InnerVolumeSpecName "kube-api-access-k2xgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:09:52.106375 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.106337    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-xks-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:09:52.106375 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.106369    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2xgz\" (UniqueName: \"kubernetes.io/projected/bfb08982-dbce-4bb0-9149-9f31f69a9911-kube-api-access-k2xgz\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:09:52.106375 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.106381    4407 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfb08982-dbce-4bb0-9149-9f31f69a9911-maas-api-openshift-ca\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:09:52.767121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.767095    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw"
Nov 02 04:09:52.767121 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.767103    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae3e5146-bw7sw" event={"ID":"bfb08982-dbce-4bb0-9149-9f31f69a9911","Type":"ContainerDied","Data":"a8f00308b491c974b98e352066fec60466874977011dc12ca53de9e533ce677d"}
Nov 02 04:09:52.767337 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:52.767149    4407 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8f00308b491c974b98e352066fec60466874977011dc12ca53de9e533ce677d"
Nov 02 04:09:56.228026 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:09:56.227990    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:04.464687 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:04.464653    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:05.188388 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:10:05.188350    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:10:05.983486 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:05.983451    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:06.253461 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:06.253431    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:06.882972 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:06.882940    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:09.061348 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:09.061314    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-p4qfk"]
Nov 02 04:10:11.552123 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:11.552090    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:11.712832 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:11.712798    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:12.508376 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:12.508344    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:12.987112 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:12.987032    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:13.178456 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:13.178428    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:13.270757 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:13.270727    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"]
Nov 02 04:10:13.270990 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:13.270965    4407 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy" containerID="cri-o://dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545" gracePeriod=30
Nov 02 04:10:13.338849 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:13.338797    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:16.612047 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:16.612015    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-p4qfk_02205620-98a5-4f95-9761-0b0461a7ed68/limitador/0.log"
Nov 02 04:10:16.824456 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:16.824428    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-dsph6_6f475a0e-297e-4609-a97c-8af20c647fb2/manager/0.log"
Nov 02 04:10:17.992196 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:17.992166    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-kh22l_e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2/manager/0.log"
Nov 02 04:10:18.036858 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.036820    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:18.215671 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.215642    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-2f5bd_a0498751-0dc7-4bc2-9076-a9c0c9411a46/manager/0.log"
Nov 02 04:10:18.513666 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.513639    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 04:10:18.569219 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569137    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-envoy\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569219 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569186    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55a35abc-96d2-419e-b183-b8eba22cb573-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a35abc-96d2-419e-b183-b8eba22cb573-istiod-ca-cert\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569219 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569209    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-kube-api-access-5msld\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-kube-api-access-5msld\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569496 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569226    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-certs\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569496 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569303    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-credential-socket\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569496 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569332    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-istio-token\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-istio-token\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569496 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569370    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/55a35abc-96d2-419e-b183-b8eba22cb573-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a35abc-96d2-419e-b183-b8eba22cb573-istio-podinfo\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569496 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569400    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-data\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569496 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569431    4407 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-socket\") pod \"55a35abc-96d2-419e-b183-b8eba22cb573\" (UID: \"55a35abc-96d2-419e-b183-b8eba22cb573\") "
Nov 02 04:10:18.569872 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569597    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-certs" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:10:18.569872 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569655    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55a35abc-96d2-419e-b183-b8eba22cb573-istiod-ca-cert" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 04:10:18.569872 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569667    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-credential-socket" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:10:18.570052 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569890    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-data" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:10:18.570052 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569904    4407 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55a35abc-96d2-419e-b183-b8eba22cb573-istiod-ca-cert\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.570052 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569908    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-socket" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:10:18.570052 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.569971    4407 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-certs\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.570438 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.570103    4407 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-credential-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.571526 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.571500    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-envoy" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 04:10:18.571697 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.571669    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-kube-api-access-5msld" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "kube-api-access-5msld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:10:18.571851 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.571833    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-istio-token" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 04:10:18.571893 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.571846    4407 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55a35abc-96d2-419e-b183-b8eba22cb573-istio-podinfo" pod "55a35abc-96d2-419e-b183-b8eba22cb573" (UID: "55a35abc-96d2-419e-b183-b8eba22cb573"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 04:10:18.671020 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.670975    4407 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-envoy\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.671020 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.671013    4407 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5msld\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-kube-api-access-5msld\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.671020 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.671026    4407 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55a35abc-96d2-419e-b183-b8eba22cb573-istio-token\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.671020 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.671034    4407 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55a35abc-96d2-419e-b183-b8eba22cb573-istio-podinfo\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.671317 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.671044    4407 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-istio-data\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.671317 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.671053    4407 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55a35abc-96d2-419e-b183-b8eba22cb573-workload-socket\") on node \"ip-10-0-131-112.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 04:10:18.887471 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.887385    4407 generic.go:356] "Generic (PLEG): container finished" podID="55a35abc-96d2-419e-b183-b8eba22cb573" containerID="dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545" exitCode=0
Nov 02 04:10:18.887471 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.887421    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" event={"ID":"55a35abc-96d2-419e-b183-b8eba22cb573","Type":"ContainerDied","Data":"dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545"}
Nov 02 04:10:18.887471 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.887432    4407 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"
Nov 02 04:10:18.887471 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.887464    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt" event={"ID":"55a35abc-96d2-419e-b183-b8eba22cb573","Type":"ContainerDied","Data":"1a4a4cf5c9227321c04f631e24ee2fc76fcc04a82703e560493bef76c215c2b0"}
Nov 02 04:10:18.887783 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.887485    4407 scope.go:122] "RemoveContainer" containerID="dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545"
Nov 02 04:10:18.899675 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.899651    4407 scope.go:122] "RemoveContainer" containerID="258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9"
Nov 02 04:10:18.907991 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.907971    4407 scope.go:122] "RemoveContainer" containerID="dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545"
Nov 02 04:10:18.908326 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:10:18.908299    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545\": container with ID starting with dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545 not found: ID does not exist" containerID="dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545"
Nov 02 04:10:18.908414 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.908336    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545"} err="failed to get container status \"dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545\": rpc error: code = NotFound desc = could not find container \"dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545\": container with ID starting with dbe601af335d2386e720af924e484154fef76afc3c95a59e05921995bf093545 not found: ID does not exist"
Nov 02 04:10:18.908414 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.908354    4407 scope.go:122] "RemoveContainer" containerID="258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9"
Nov 02 04:10:18.908712 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:10:18.908689    4407 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9\": container with ID starting with 258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9 not found: ID does not exist" containerID="258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9"
Nov 02 04:10:18.908756 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:18.908720    4407 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9"} err="failed to get container status \"258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9\": rpc error: code = NotFound desc = could not find container \"258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9\": container with ID starting with 258b23c0f14ea54e4822ad0dbdb0a0b3f66da42293137b3850a484ae343171a9 not found: ID does not exist"
Nov 02 04:10:19.057200 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:19.057163    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"]
Nov 02 04:10:19.059801 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:19.059783    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-858236-openshift-default-74fcd64bf5-dvdrt"]
Nov 02 04:10:19.106640 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:19.106592    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-rhpmb_e4694bf5-7e68-4fff-8908-53345db86ffc/server/0.log"
Nov 02 04:10:19.313423 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:19.313394    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-qj42d_feaad75e-40cc-46a5-83f7-586c25f24ab4/manager/0.log"
Nov 02 04:10:20.350085 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:20.350053    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" path="/var/lib/kubelet/pods/55a35abc-96d2-419e-b183-b8eba22cb573/volumes"
Nov 02 04:10:30.532608 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:30.532576    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-kh22l_e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2/manager/0.log"
Nov 02 04:10:30.792386 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:30.792306    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-2f5bd_a0498751-0dc7-4bc2-9076-a9c0c9411a46/manager/0.log"
Nov 02 04:10:31.811789 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:31.811765    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-rhpmb_e4694bf5-7e68-4fff-8908-53345db86ffc/server/0.log"
Nov 02 04:10:32.015110 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:32.015080    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-qj42d_feaad75e-40cc-46a5-83f7-586c25f24ab4/manager/0.log"
Nov 02 04:10:33.113825 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:33.113790    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-57dc95b55c-tm7zm_f58066e0-8b2a-4916-9895-f88e667a8237/maas-api/0.log"
Nov 02 04:10:33.679139 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:33.679103    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-60f3081f-rpwtj_b1c97893-5ded-49f6-9781-aa316c489320/revoke-keys/0.log"
Nov 02 04:10:34.102767 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:34.102735    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-ae3e5146-bw7sw_bfb08982-dbce-4bb0-9149-9f31f69a9911/revoke-keys/0.log"
Nov 02 04:10:35.188020 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:10:35.187978    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:10:37.693956 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:37.693927    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-p4qfk_02205620-98a5-4f95-9761-0b0461a7ed68/limitador/0.log"
Nov 02 04:10:37.900369 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:37.900263    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-dsph6_6f475a0e-297e-4609-a97c-8af20c647fb2/manager/0.log"
Nov 02 04:10:39.213876 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:39.213845    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66599bc4bb-sx4rr_f4c44847-af85-4769-9ca1-1b03e7fbdf4e/kube-auth-proxy/0.log"
Nov 02 04:10:39.634968 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:39.634940    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-nmpc6_d890d798-0f37-4e21-a517-c382a56b9ca3/istio-proxy/0.log"
Nov 02 04:10:41.323950 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:41.323922    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-kmlfj_9b79df1c-7f4d-4e26-bd41-157a2e656c23/main/0.log"
Nov 02 04:10:41.779561 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:41.779521    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:42.292322 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:42.292284    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b88xqfv_a1c469a1-a23a-47df-8dbc-8d9e642efc40/main/0.log"
Nov 02 04:10:42.551865 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:42.551780    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-hxn75_6be96fa7-288b-40a2-8507-5b4da7da07a7/main/0.log"
Nov 02 04:10:42.772945 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:42.772918    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-lpq5s_7cd159c5-459e-4cda-8d90-f9264901044b/main/0.log"
Nov 02 04:10:45.262931 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:45.262902    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:52.510332 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.510251    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:52.564509 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.564479    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rbmzv/must-gather-f5h92"]
Nov 02 04:10:52.564894 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.564880    4407 cpu_manager.go:414] "containerMap: removing container" podUID="bfb08982-dbce-4bb0-9149-9f31f69a9911" containerName="revoke-keys"
Nov 02 04:10:52.564947 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.564894    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfb08982-dbce-4bb0-9149-9f31f69a9911" containerName="revoke-keys"
Nov 02 04:10:52.564947 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.564911    4407 cpu_manager.go:414] "containerMap: removing container" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy"
Nov 02 04:10:52.564947 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.564916    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy"
Nov 02 04:10:52.565051 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.564978    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy"
Nov 02 04:10:52.565051 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.564989    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfb08982-dbce-4bb0-9149-9f31f69a9911" containerName="revoke-keys"
Nov 02 04:10:52.565051 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.565040    4407 cpu_manager.go:414] "containerMap: removing container" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy"
Nov 02 04:10:52.565051 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.565045    4407 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy"
Nov 02 04:10:52.565180 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.565118    4407 memory_manager.go:365] "RemoveStaleState removing state" podUID="55a35abc-96d2-419e-b183-b8eba22cb573" containerName="istio-proxy"
Nov 02 04:10:52.568097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.568080    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbmzv/must-gather-f5h92"
Nov 02 04:10:52.624536 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.624505    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbmzv/must-gather-f5h92"]
Nov 02 04:10:52.673146 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.673108    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbmzv\"/\"openshift-service-ca.crt\""
Nov 02 04:10:52.673146 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.673108    4407 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rbmzv\"/\"default-dockercfg-c7lwr\""
Nov 02 04:10:52.673146 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.673140    4407 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbmzv\"/\"kube-root-ca.crt\""
Nov 02 04:10:52.689066 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.689034    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v9sm\" (UniqueName: \"kubernetes.io/projected/ceb83224-8c99-4d31-bee9-43c6aeb2256a-kube-api-access-5v9sm\") pod \"must-gather-f5h92\" (UID: \"ceb83224-8c99-4d31-bee9-43c6aeb2256a\") " pod="openshift-must-gather-rbmzv/must-gather-f5h92"
Nov 02 04:10:52.689227 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.689084    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ceb83224-8c99-4d31-bee9-43c6aeb2256a-must-gather-output\") pod \"must-gather-f5h92\" (UID: \"ceb83224-8c99-4d31-bee9-43c6aeb2256a\") " pod="openshift-must-gather-rbmzv/must-gather-f5h92"
Nov 02 04:10:52.692677 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.692657    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:52.790322 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.790249    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v9sm\" (UniqueName: \"kubernetes.io/projected/ceb83224-8c99-4d31-bee9-43c6aeb2256a-kube-api-access-5v9sm\") pod \"must-gather-f5h92\" (UID: \"ceb83224-8c99-4d31-bee9-43c6aeb2256a\") " pod="openshift-must-gather-rbmzv/must-gather-f5h92"
Nov 02 04:10:52.790322 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.790316    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ceb83224-8c99-4d31-bee9-43c6aeb2256a-must-gather-output\") pod \"must-gather-f5h92\" (UID: \"ceb83224-8c99-4d31-bee9-43c6aeb2256a\") " pod="openshift-must-gather-rbmzv/must-gather-f5h92"
Nov 02 04:10:52.790680 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.790664    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ceb83224-8c99-4d31-bee9-43c6aeb2256a-must-gather-output\") pod \"must-gather-f5h92\" (UID: \"ceb83224-8c99-4d31-bee9-43c6aeb2256a\") " pod="openshift-must-gather-rbmzv/must-gather-f5h92"
Nov 02 04:10:52.847745 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.847709    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v9sm\" (UniqueName: \"kubernetes.io/projected/ceb83224-8c99-4d31-bee9-43c6aeb2256a-kube-api-access-5v9sm\") pod \"must-gather-f5h92\" (UID: \"ceb83224-8c99-4d31-bee9-43c6aeb2256a\") " pod="openshift-must-gather-rbmzv/must-gather-f5h92"
Nov 02 04:10:52.881674 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:52.881647    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbmzv/must-gather-f5h92"
Nov 02 04:10:53.025283 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:53.025248    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:53.177663 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:53.177563    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:10:53.251021 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:53.250996    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbmzv/must-gather-f5h92"]
Nov 02 04:10:53.252272 ip-10-0-131-112 kubenswrapper[4407]: W1102 04:10:53.252241    4407 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podceb83224_8c99_4d31_bee9_43c6aeb2256a.slice/crio-bcde6ad82005352f52ba0d7d680c343d74a4ee71a973839704984041444b993a WatchSource:0}: Error finding container bcde6ad82005352f52ba0d7d680c343d74a4ee71a973839704984041444b993a: Status 404 returned error can't find the container with id bcde6ad82005352f52ba0d7d680c343d74a4ee71a973839704984041444b993a
Nov 02 04:10:53.254724 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:53.254709    4407 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 04:10:54.052859 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:54.052823    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbmzv/must-gather-f5h92" event={"ID":"ceb83224-8c99-4d31-bee9-43c6aeb2256a","Type":"ContainerStarted","Data":"bcde6ad82005352f52ba0d7d680c343d74a4ee71a973839704984041444b993a"}
Nov 02 04:10:55.060641 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:55.060579    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbmzv/must-gather-f5h92" event={"ID":"ceb83224-8c99-4d31-bee9-43c6aeb2256a","Type":"ContainerStarted","Data":"1a2b25c517a0c4e965cd02d70cad64d8ca3edfbafc11b99c0219f29efc260dfa"}
Nov 02 04:10:55.061056 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:55.060645    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbmzv/must-gather-f5h92" event={"ID":"ceb83224-8c99-4d31-bee9-43c6aeb2256a","Type":"ContainerStarted","Data":"e3e93f68e3c7b7ea532de159020c96421cac01c340ddaf069e04461bc3aea26c"}
Nov 02 04:10:55.176212 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:55.176099    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-rbmzv/must-gather-f5h92" podStartSLOduration=1.844095817 podStartE2EDuration="3.176084006s" totalImagesPullingTime="1.331988189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 04:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:10:55.175057005 +0000 UTC m=+4998.970405572" watchObservedRunningTime="2026-11-02 04:10:55.176084006 +0000 UTC m=+4998.971432570"
Nov 02 04:10:56.052455 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:56.052420    4407 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"]
Nov 02 04:10:56.103224 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:56.103187    4407 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60f3081f-rpwtj"]
Nov 02 04:10:56.350652 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:56.350546    4407 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1c97893-5ded-49f6-9781-aa316c489320" path="/var/lib/kubelet/pods/b1c97893-5ded-49f6-9781-aa316c489320/volumes"
Nov 02 04:10:56.549662 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:10:56.549602    4407 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log" limit=1024
Nov 02 04:11:02.686659 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:02.680944    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zs4kx_367d8f5b-1f4f-4d94-a9aa-7b007cd260a5/global-pull-secret-syncer/0.log"
Nov 02 04:11:04.569673 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:04.569614    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b5wvk_ee88ccb5-4a04-431a-8963-7b8c7393604a/konnectivity-agent/0.log"
Nov 02 04:11:04.702019 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:04.701947    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-kh22l_e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2/manager/0.log"
Nov 02 04:11:04.957611 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:04.957522    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-2f5bd_a0498751-0dc7-4bc2-9076-a9c0c9411a46/manager/0.log"
Nov 02 04:11:05.231283 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:11:05.231170    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:11:05.798310 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:05.798257    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-rhpmb_e4694bf5-7e68-4fff-8908-53345db86ffc/server/0.log"
Nov 02 04:11:05.831057 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:05.831023    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-112.us-west-1.compute.internal_9fdb79d33d08466df57752003cd13435/haproxy/0.log"
Nov 02 04:11:06.839890 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:06.839853    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-qj42d_feaad75e-40cc-46a5-83f7-586c25f24ab4/manager/0.log"
Nov 02 04:11:10.245071 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:10.245038    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-kh22l_e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2/manager/0.log"
Nov 02 04:11:10.507036 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:10.504846    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-2f5bd_a0498751-0dc7-4bc2-9076-a9c0c9411a46/manager/0.log"
Nov 02 04:11:11.374110 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.374064    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-rhpmb_e4694bf5-7e68-4fff-8908-53345db86ffc/server/0.log"
Nov 02 04:11:11.620202 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.620168    4407 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"]
Nov 02 04:11:11.627601 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.627517    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.681269 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.681237    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"]
Nov 02 04:11:11.685762 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.685726    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-lib-modules\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.685947 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.685867    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-sys\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.685947 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.685895    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-podres\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.685947 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.685921    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-proc\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.686136 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.685952    4407 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6vhj\" (UniqueName: \"kubernetes.io/projected/5b8be219-f36e-40bb-836d-40aabc670daf-kube-api-access-d6vhj\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.786587 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786547    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-lib-modules\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.786810 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786722    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-sys\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.786810 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786751    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-podres\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.786810 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786763    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-lib-modules\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.786810 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786791    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-proc\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.787097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786825    4407 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6vhj\" (UniqueName: \"kubernetes.io/projected/5b8be219-f36e-40bb-836d-40aabc670daf-kube-api-access-d6vhj\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.787097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786845    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-sys\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.787097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786849    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-podres\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.787097 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.786906    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b8be219-f36e-40bb-836d-40aabc670daf-proc\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.845266 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.845232    4407 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6vhj\" (UniqueName: \"kubernetes.io/projected/5b8be219-f36e-40bb-836d-40aabc670daf-kube-api-access-d6vhj\") pod \"perf-node-gather-daemonset-6t752\" (UID: \"5b8be219-f36e-40bb-836d-40aabc670daf\") " pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:11.943614 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:11.943532    4407 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:12.343665 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:12.343617    4407 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"]
Nov 02 04:11:12.395363 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:12.395343    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-qj42d_feaad75e-40cc-46a5-83f7-586c25f24ab4/manager/0.log"
Nov 02 04:11:13.192497 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:13.192448    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752" event={"ID":"5b8be219-f36e-40bb-836d-40aabc670daf","Type":"ContainerStarted","Data":"ee470726b278b05d5289ca88f42a630be142e7af8abc65ae131c875d74673f2b"}
Nov 02 04:11:13.192725 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:13.192517    4407 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752" event={"ID":"5b8be219-f36e-40bb-836d-40aabc670daf","Type":"ContainerStarted","Data":"7c20f538a73e64c584b136ba4f88feaab9fc73a6bf196357e8d34e49884caf77"}
Nov 02 04:11:13.192816 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:13.192755    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:13.310012 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:13.309951    4407 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752" podStartSLOduration=2.309935998 podStartE2EDuration="2.309935998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 04:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 04:11:13.308787157 +0000 UTC m=+5017.104135724" watchObservedRunningTime="2026-11-02 04:11:13.309935998 +0000 UTC m=+5017.105284564"
Nov 02 04:11:13.649985 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:13.649958    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-p4qfk_02205620-98a5-4f95-9761-0b0461a7ed68/limitador/0.log"
Nov 02 04:11:14.103827 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:14.103773    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-dsph6_6f475a0e-297e-4609-a97c-8af20c647fb2/manager/0.log"
Nov 02 04:11:15.645272 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:15.645236    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-kh22l_e39eb805-c6c1-4c36-8f29-9e9c0b7ac0f2/manager/0.log"
Nov 02 04:11:15.892827 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:15.892798    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-2f5bd_a0498751-0dc7-4bc2-9076-a9c0c9411a46/manager/0.log"
Nov 02 04:11:16.654048 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:16.654023    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d3eea3a-2a6c-4acf-b811-e261852d0a58/alertmanager/0.log"
Nov 02 04:11:16.753694 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:16.753664    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-rhpmb_e4694bf5-7e68-4fff-8908-53345db86ffc/server/0.log"
Nov 02 04:11:17.066159 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:17.066133    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d3eea3a-2a6c-4acf-b811-e261852d0a58/config-reloader/0.log"
Nov 02 04:11:17.483045 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:17.482965    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d3eea3a-2a6c-4acf-b811-e261852d0a58/kube-rbac-proxy-web/0.log"
Nov 02 04:11:17.697639 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:17.697599    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-qj42d_feaad75e-40cc-46a5-83f7-586c25f24ab4/manager/0.log"
Nov 02 04:11:17.896114 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:17.896079    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d3eea3a-2a6c-4acf-b811-e261852d0a58/kube-rbac-proxy/0.log"
Nov 02 04:11:18.308662 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:18.308612    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d3eea3a-2a6c-4acf-b811-e261852d0a58/kube-rbac-proxy-metric/0.log"
Nov 02 04:11:18.721614 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:18.721529    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d3eea3a-2a6c-4acf-b811-e261852d0a58/prom-label-proxy/0.log"
Nov 02 04:11:19.135246 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:19.135218    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d3eea3a-2a6c-4acf-b811-e261852d0a58/init-config-reloader/0.log"
Nov 02 04:11:19.207476 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:19.207449    4407 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rbmzv/perf-node-gather-daemonset-6t752"
Nov 02 04:11:21.912534 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:21.912504    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66599bc4bb-sx4rr_f4c44847-af85-4769-9ca1-1b03e7fbdf4e/kube-auth-proxy/0.log"
Nov 02 04:11:22.340559 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:22.340529    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-nmpc6_d890d798-0f37-4e21-a517-c382a56b9ca3/istio-proxy/0.log"
Nov 02 04:11:24.572011 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:24.571984    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8dft_0eec203a-7c01-43d3-ae10-0c6e423f73e4/node-exporter/0.log"
Nov 02 04:11:25.026148 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:25.026118    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8dft_0eec203a-7c01-43d3-ae10-0c6e423f73e4/kube-rbac-proxy/0.log"
Nov 02 04:11:25.437527 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:25.437448    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8dft_0eec203a-7c01-43d3-ae10-0c6e423f73e4/init-textfile/0.log"
Nov 02 04:11:27.104846 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:27.104817    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b9f9767f-d787-4e76-bea2-26669d01a593/prometheus/0.log"
Nov 02 04:11:27.517119 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:27.517089    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b9f9767f-d787-4e76-bea2-26669d01a593/config-reloader/0.log"
Nov 02 04:11:27.930047 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:27.929968    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b9f9767f-d787-4e76-bea2-26669d01a593/thanos-sidecar/0.log"
Nov 02 04:11:28.341249 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:28.341225    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b9f9767f-d787-4e76-bea2-26669d01a593/kube-rbac-proxy-web/0.log"
Nov 02 04:11:28.753885 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:28.753857    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b9f9767f-d787-4e76-bea2-26669d01a593/kube-rbac-proxy/0.log"
Nov 02 04:11:29.168797 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:29.168723    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b9f9767f-d787-4e76-bea2-26669d01a593/kube-rbac-proxy-thanos/0.log"
Nov 02 04:11:29.582556 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:29.582527    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b9f9767f-d787-4e76-bea2-26669d01a593/init-config-reloader/0.log"
Nov 02 04:11:30.184915 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:30.184887    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-p4qfk_02205620-98a5-4f95-9761-0b0461a7ed68/limitador/0.log"
Nov 02 04:11:30.431131 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:30.431099    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-dsph6_6f475a0e-297e-4609-a97c-8af20c647fb2/manager/0.log"
Nov 02 04:11:31.252646 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:31.252600    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bddbb5fbf-rdj2j_e1ebd107-87fa-4417-98bc-68c9bcb531e6/telemeter-client/0.log"
Nov 02 04:11:31.664559 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:31.664495    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bddbb5fbf-rdj2j_e1ebd107-87fa-4417-98bc-68c9bcb531e6/reload/0.log"
Nov 02 04:11:32.075365 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:32.075335    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bddbb5fbf-rdj2j_e1ebd107-87fa-4417-98bc-68c9bcb531e6/kube-rbac-proxy/0.log"
Nov 02 04:11:35.188672 ip-10-0-131-112 kubenswrapper[4407]: E1102 04:11:35.188617    4407 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vzzk7" containerName="extractor"
Nov 02 04:11:35.478741 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:35.478659    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66599bc4bb-sx4rr_f4c44847-af85-4769-9ca1-1b03e7fbdf4e/kube-auth-proxy/0.log"
Nov 02 04:11:35.902528 ip-10-0-131-112 kubenswrapper[4407]: I1102 04:11:35.902493    4407 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-nmpc6_d890d798-0f37-4e21-a517-c382a56b9ca3/istio-proxy/0.log"